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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION 
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SUBJECT ;  Report  on  the  Implementation  of  Interoperability  and  Information 

Assurance  Policies  for  Acquisition  of  DoD  Weapon  Systems  (Report 
No.  D-2003-011) 


We  are  providing  this  report  for  review  and  comment.  This  report  is  the  first  in 
a  series  of  reports  that  discuss  the  overall  implementation  of  interoperability  and 
information  assurance  policies  for  the  acquisition  of  DoD  systems.  This  report 
addresses  the  implementation  of  those  policies  within  the  Office  of  the  Secretary  of 
Defense  and  the  Defense  agencies.  We  considered  management  comments  on  a  draft  of 
this  report  when  preparing  the  final  report. 

DoD  Directive  7650.3  requires  that  all  recommendations  be  resolved  promptly. 
As  a  result  of  management  comments,  we  revised  Recommendations  1.,  3.a.,  and  4.b. 
We  redirected  Recommendations  1.,  2.,  3. a.,  3.b.,  4.a.  and  4.b.  to  clarify  our  intention 
and  request  comments  on  how  management  plans  to  implement  the  recommendations. 
Therefore,  we  request  that  the  Co-Chair,  Interoperability  Senior  Review  Panel  provide 
comments  by  December  16,  2002. 

We  appreciate  the  courtesies  extended  to  the  staff.  (Questions  should  be  directed 
to  Mr.  John  E.  Meling  at  (703)  604-9091  (DSN  664-9091)  (jmelmg@dodig.osd.mil)  or 
Mr.  Jack  D.  Snider  at  (703)  604-9087  (DSN  664-9087)  (jsnider@dodig.osd.mil).  See 
Appendix  J  for  the  report  distribution.  The  team  members  are  listed  inside  the  back 
cover. 


David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 
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Implementation  of  Interoperability  and 
Information  Assurance  Policies  for 
Acquisition  of  DoD  Weapon  Systems 


Executive  Summary 


Who  Should  Read  This  Report  and  Why?  Policy  makers,  milestone  deeision 
makers,  eombat  and  materiel  developers,  and  testers  responsible  for  interoperability  and 
information  assuranee  requirements  of  DoD  weapon  systems  should  be  interested  in  this 
report.  This  report  addresses  the  importanee  of  adhering  to  DoD  interoperability  and 
information  assuranee  polieies  to  reduee  the  risk  of  DoD  weapon  systems  not  being 
interoperable  and  able  to  exehange  information  in  a  seeure  manner  with  other  DoD  and 
allied  systems. 

Background.  This  report  is  the  first  in  a  series  of  reports  on  the  implementation  of 
interoperability  and  information  assuranee  polieies  for  the  aequisition  of  DoD  weapon 
systems.  Other  reports  in  the  series  will  address  how  effeetively  the  Army,  Navy, 

Air  Foree,  and  Unified  Commands  implement  those  polieies.  Interoperability  and 
information  assuranee  polieies  inelude  the  Joint  Vision  2020  and  the  Global  information 
Grid  eapstone  requirement  doeument. 

Results.  The  Department  faees  a  diffieult  ehallenge  in  aehieving  interoperability 
between  DoD  systems  and  needs  eongruent  and  effeetive  meehanisms  to  measure  and 
oversee  its  progress.  Without  eonsistent  guidanee  that  makes  eombat  and  materiel 
developers  analyze  programs  using  an  operational  arehiteeture  view,  the  DoD  is  at  risk 
of  developing  systems  that  operate  independently  of  other  systems  and  of  not  fully 
realizing  the  benefits  of  interoperable  DoD  systems  to  satisfy  the  needs  of  the 
warfighter  as  outlined  in  Joint  Vision  2020.  Implementing  a  proeess  that  timely 
integrates  revisions  for  interoperability  and  information  assuranee  polieies  into  the 
applieable  DoD  and  Chairman  of  the  Joint  Chiefs  of  Staff  interoperability  and 
information  assuranee  polieies;  establishing  eriteria  and  proeedures  for  plaeing  DoD 
systems  on  the  Interoperability  Wateh  List;  eomparing  the  operational  requirements 
doeuments  (ORDs)  of  proposed  DoD  systems  against  the  other  ORDs  in  the  related 
mission  area  arehiteeture;  updating  the  Joint  Command,  Control,  Communieations, 
Computers,  and  Intelligenee  Program  Assessment  Tool  (JCPAT)  database  and 
eontr oiling  user  aeeess;  and  defining  and  implementing  a  plan  to  test  eritieal  operational 
issues  for  DoD  systems  in  the  Global  Information  Grid  should  better  enable  DoD  to 
implement  interoperability  and  information  assuranee  polieies.  (See  the  Finding  seetion 
of  this  report  for  the  detailed  reeommendations.) 

Management  Comments  and  Audit  Response.  We  reeeived  comments  from  the 
Direetor,  Interoperability,  Offiee  of  the  Under  Seeretary  of  Defense  for  Aequisition, 
Teehnology,  and  Logisties;  the  Direetor,  Arehiteeture  and  Interoperability,  Offiee  of 
the  Assistant  Seeretary  of  Defense  (Command,  Control,  Communieations,  and 
Intelligenee)  (ASD[C3I]);  the  Inspeetor  General,  Offiee  of  the  Commander,  U.S.  Joint 


Forces  Command  (USJFCOM);  the  Direetor,  Operational  Test  and  Evaluation;  the 
Inspeetor  General,  Defense  Information  Systems  Ageney  (DISA);  the  Direetor,  Joint 
Staff;  and  the  Co-Chair,  Interoperability  Senior  Review  Panel.* 

The  Direetor,  Interoperability  neither  eoneurred  nor  noneoneurred  with  the 
reeommendation  eoneerning  timely  revisions  of  interoperability  and  information 
assuranee  polieies;  however,  he  suggested  revisions  to  the  reeommendation,  whieh  we 
made.  The  Direetor,  Arehiteeture  and  Interoperability  eoneurred  with  the 
reeommendation  eoneerning  timely  revisions  of  interoperability  and  information 
assuranee  polieies  and  neither  eoneurred  nor  noneoneurred  widi  the  reeommendation 
eoneerning  the  implementation  of  the  Interoperability  Wateh  List.  The  Inspeetor 
General,  USJFCOM  stated  that  USJFCOM  neither  eoneurred  nor  noneoneurred  with 
the  reeommendation  eoneerning  the  eomparison  of  ORDs;  however,  he  suggested 
revisions  to  the  reeommendation,  whieh  we  made.  The  Inspeetor  General  added  that 
the  USJFCOM  is  eurrently  not  funded  or  resoureed  to  perform  the  in-depth  review  of 
the  ORD,  as  reeommended.  The  Direetor,  Operational  Test  and  Evaluation  partially 
eoneurred  with  the  reeommendations  eoneerning  the  implementation  of  the 
Interoperability  Wateh  List  and  the  testing  of  eritieal  operational  issues.  The  Direetor, 
Joint  Staff  eoneurred  with  the  report,  subjeet  to  the  ineorporation  of  his  eomments  on 
the  recommendations  eoneerning  interoperability  and  information  assuranee  polieies, 
the  Interoperability  Wateh  List,  and  limiting  JCPAT  aeeess.  The  Direetor  also  stated 
that  the  DoD  is  not  effeetively  struetured  to  effeet  the  organizing,  training,  and 
equipping  of  joint  eapabilities.  Further,  he  stated  that  a  joint  proeess  was  not 
established  to  delineate  who  is  responsible  and  aeeountable  for  developing  and 
aequiring  joint  eommand  and  eontrol  systems  and  integrating  eapabilities.  The 
Co-Chair,  Interoperability  Senior  Review  Panel  agreed  with  the  need  for  timely 
revisions  of  interoperability  and  information  assuranee  polieies,  the  implementation  of 
the  Interoperability  Wateh  List,  the  eomparison  of  ORDs,  the  use  of  qualifieation 
analysis  and  predietive  tools,  and  the  testing  of  eritieal  operational  issues.  Further,  the 
Co-Chair  supported  the  position  of  the  Joint  Staff  regarding  the  JCPAT.  (See  the 
Finding  seetion  of  this  report  for  a  diseussion  of  the  management  comments  and  the 
Management  Comments  seetion  of  the  report  for  the  eomplete  text  of  the  eomments.) 

Beeause  we  reeeived  varied  comments  from  the  senior  leadership  of  the  Interoperability 
Senior  Review  Panel  to  the  reeommendations,  often  without  a  eoordinated  eorreetive 
aetion  plan,  we  redireeted  those  reeommendations  to  the  Co-Chair,  Interoperability 
Senior  Review  Panel  and  request  that  he  comment  on  how  the  Panel  will  implement  a 
proeess  that  timely  integrates  revisions  for  interoperability  and  information  assuranee 
polieies;  will  establish  a  elearly  defined  proeess  and  eriteria  for  the  Interoperability 
Wateh  List;  will  employ  a  proeess  and  resourees,  ineluding  quantifieation  analysis  and 
predietive  tools,  to  eompare  the  ORDs  of  proposed  DoD  systems  against  the  other 
ORDs  in  the  related  mission  area  arehiteeture;  will  update  or  arehive  JCPAT 
doeuments;  and  will  limit  JCPAT  aeeess.  Therefore,  we  request  that  the  Co-Chair, 
Interoperability  Senior  Review  Panel  provide  comments  on  this  final  report  by 
Deeember  16,  2002. 


*The  Interoperability  Senior  Review  Panel  is  composed  of  senior  leaders  from  the  Offices  of  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics;  the  ASD(Ch);  the  Joint  Staff;  the 
USJFCOM;  the  Director  for  Programs,  Analysis,  and  Evaluation;  and  the  Director,  Operational  Test  and 
Evaluation. 
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Background 


This  report  is  the  first  in  a  series  of  reports  on  the  implementation  of 
interoperability  and  information  assuranee  polieies  for  the  aequisition  of  DoD 
weapon  systems.  This  report  addresses  the  implementation  of  those  polieies  by 
the  Offiee  of  the  Seeretary  of  Defense  and  Defense  ageneies,  the  interoperability 
requirements  proeess,  and  the  oversight  thereof  Subsequent  reports  will 
diseuss  the  adequaey  of  interoperability  key  performanee  parameters  (KPPs)  and 
the  interoperability  eertifieation  proeess  for  DoD  systems,  ineluding  national 
seeurity  systems  in  the  Army,  Navy,  Air  Foree,  and  unified  eombatant 
commands.  Appendix  B  provides  definitions  of  technical  terms  used  in  this 
report. 

Chairman  of  the  Joint  Chiefs  of  Staff  Testimony  on  the  President’s 
Proposed  Defense  Program  for  FYs  2003  to  2007.  On  February  5,  2002, 
General  Myers,  the  Chairman  of  the  Joint  Chiefs  of  Staff,  testified  before  the 
U.S.  Senate  Committee  on  Armed  Services  on  interoperability.  General  Myers 
described  how  Navy,  Air  Force,  and  Marine  Corps  systems  shared  information 
to  execute  combat  operations  in  Afghanistan.  He  testified  that: 

To  fulfill  our  range  of  commitments  and  protect  our  global  interests, 
we  must  make  the  investments  necessary  to  maintain  the  quality  of 
our  force  while  preparing  for  future  challenges  of  the  2P‘  century. 

The  best  means  of  accomplishing  these  goals,  in  my  mind,  are  to, 
one,  improve  our  joint  war-fighting  capabilities,  and  two,  to 
transform  the  armed  forces  of  America  into  a  2P‘  century  force. 

General  Myers  further  testified  that  DoD  should  conceive,  design,  and  produce 
new  systems  with  joint  warfighting  requirements  in  mind.  DoD  needs  to  view 
new  systems  as  interchangeaWe  modules  that  can  be  used  in  any  situation  and  in 
any  command  arrangement.  General  Myers  believed  the  area  with  the  greatest 
potential  is  in  command,  control,  communications,  computers,  intelligence, 
surveillance  and  reconnaissance. 

Quadrennial  Defense  Review.  On  September  2001,  the  Quadrennial  Defense 
Review  report  stated  that  achieving  the  objectives  of  the  defense  strategy 
requires  the  transformation  of  the  U.S.  Armed  Forces.  Two  of  the  six  critical 
operational  goals  for  the  DoD  transformational  efforts  relate  to  information 
assurance  and  interoperability. 

•  Assuring  that,  in  the  face  of  attack,  information  systems  conduct 
effective  information  operations. 

•  Leveraging  information  technology  and  innovative  concepts  to 
develop  interoperable,  joint  command,  control,  communications, 
computers,  intelligence,  surveillance,  and  reconnaissance 
architectures  and  capability  that  includes  a  tailorable  joint  operational 
picture. 
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Additionally,  as  stated  in  the  Quadrennial  Defense  Review  report, 


To  support  joint  and  combined  command  and  control  and  to  enable  a 
common  relevant  operational  picture  of  the  battlespace,  the 
Department  will  enhance  end-to-end  interoperable  communications  for 
secure  plaiming  and  operations.  These  communications  will  provide 
shared  situational  awareness  and  integration  of  joint  fires,  maneuver, 
and  intelligence.  They  must  be  interoperable  across  all  components 
and  tailorable  for  coalition  operations  with  other  countries. 

Joint  Vision  2020.  On  May  30,  2000,  the  Chairman  of  the  Joint  Chiefs  of  Staff 
signed  Joint  Vision  2020,  whieh  addressed  the  eoneept,  design,  and  produetion 
of  systems  with  joint  warfighting  requirements.  Joint  Vision  2020  deseribes  in 
broad  terms  a  future  joint  foree  whose  operational  eapabilities  will  be  required 
to  sueeeed  aeross  the  full  range  of  military  operations  and  aeeomplish  missions 
in  the  year  2020  and  beyond.  Joint  Vision  2020  states  that  interoperability  is  a 
mandate  for  the  future  joint  foree  espeeially  for  eommunieations,  common 
logistics  items,  and  information  sharing.  Information  systems  and  equipment 
that  enable  a  common,  relevant  operational  picture  must  work  from  shared 
networks  that  can  be  accessed  by  any  appropriately  cleared  participant.  As  an 
example.  Appendix  C  details  the  action  plan  that  the  Director  for 
Interoperability,  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  is  planning  to  execute  for  command  and  control 
systems. 

Another  tenet  of  Joint  Vision  2020  is  to  attain  information  superiority. 
Information  superiority  gets  the  right  information  to  the  right  people,  at  the  right 
time,  and  in  the  right  format,  resulting  in  a  vastly  improved  sWed 
understanding  of  the  situation.  Joint  Vision  2020  emphasizes  the  following 
information  superiority  goals: 

•  implement  effective  programming  for  establishing  information 
assurance  and  critical  infrastructure  protection,  and 

•  build  a  coherent  Global  Information  Grid  (GIG). 

Global  Information  Grid.  In  November  1999,  the  Joint  Chiefs  of  Staff 
assigned  the  Commander,  U.S.  Joint  Forces  Command  (USJFCOM)  the  task  of 
preparing  the  capstone  requirements  document  (CRD)  for  the  GIG.  On 
August  30,  2001,  the  Joint  Requirements  Oversight  Council  approved  the  CRD, 
which  describes  the  overarching  information  capability  requirements  for  a 
globally  interconnected,  end-to-end,  interoperable,  and  secured 
system-of-systems  that  would  support  the  Secretary  of  Defense,  warfighters, 
DoD  personnel,  the  intelligence  community,  policy  makers,  and  non-DoD  users 
at  all  levels  involved  in  military  and  nonmilitary  operations.  Appendix  D 
provides  a  detailed  description  of  the  GIG. 

Common  Interoperable  and  Secnre  Systems.  To  attain  Joint  Vision  2020  and 
GIG  compliance  and  reduce  the  risk  of  building  stand-alone  or  “stovepipe” 
systems,  the  Defense  agencies  and  the  Military  Departments  are  required  to 
develop  and  retrofit  DoD  systems  into  common  interoperable  and  secure 
systems.  Information  assurance  is  required  to  protect,  defend,  and  ensure  the 
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availability  of  information  and  information  systems.  Interoperability  is  required 
to  provide  the  ability  of  systems  to  exehange  data  effeetively.  Together, 
information  assuranee  and  interoperability  eomprise  the  basis  for  aehieving 
network  eentrie  warfare,  information  superiority,  deeision  superiority,  and  full 
speetrum  dominanee,  as  defined  in  Joint  Vision  2020.  The  DoD  faees  a 
eomplex  ehallenge  in  aehieving  an  effeetive  eoexistenee  between  interoperability 
and  information  assuranee  within  the  eontext  of  Joint  Vision  2020  and  the  GIG. 
Appendix  E  shows  the  multiple  faetors  affeeting  the  implementation  of 
interoperability  requirements  in  the  weapon  system  development  proeess. 


Objectives 


The  primary  audit  objeetive  was  to  evaluate  whether  the  Offiee  of  the  Seeretary 
of  Defense  and  the  Defense  ageneies  were  effeetively  implementing  DoD 
interoperability  and  information  assuranee  polieies.  Subsequent  reports  will 
diseuss  the  adequaey  of  interoperability  KPPs  and  the  interoperability 
eertifieation  proeess  for  DoD  systems,  ineluding  national  seeurity  systems  in  the 
Army,  Navy,  Air  Foree,  and  unified  eombatant  eommands.  See  Appendix  A 
for  a  diseussion  of  the  audit  seope  and  methodology,  and  prior  eoverage  related 
to  the  audit  objeetives. 
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Adequacy  of  the  Interoperability  and 
Information  Assurance  Process 

The  Department  faees  a  diffieult  ehallenge  in  aehieving  interoperability 
between  DoD  systems  and  needs  eongruent  and  effeetive  meehanisms  to 
measure  and  oversee  its  progress.  The  Department  did  not  have 
fundamental  meehanisms  that  are  eonsistently  applied  or  established 
beeause; 

•  The  Assistant  Seeretary  of  Defense  (Command,  Control, 
Communieations,  and  Intelligenee)  (ASD[C3I])  had 
interoperability  polieies  in  the  1992  version  of  DoD 
Instruetion  4630.8,  “Proeedures  for  Compatibility, 
Interoperability,  and  Integration  of  Command,  Control, 
Communieations,  and  Intelligenee  (C3I)  Systems,”  that  were 
ineonsistent  with  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruetion  62 12.0 IB,  “Interoperability  and  Supportability  of 
National  Seeurity  Systems,  and  Information  Teehnology 
Systems,”  May  8,  2000.  During  the  audit,  on  May  2,  2002, 
DoD  updated  DoD  Instruetion  4630.8,  “Proeedures  for 
Interoperability  and  Supportability  of  Information  Teehnology 
(IT)  and  National  Seeurity  Systems  (NSS),”‘  to  reetify  the 
poliey  implementation  ineonsisteneies  in  the  earlier  version. 

•  The  Commander,  USJFCOM  did  not  analyze  interoperability 
and  information  assuranee  requirements  in  the  operational  and 
system  arehiteetures  for  DoD  systems  as  part  of  the 
interoperability  eertifieation  proeess^  for  operational 
requirements  doeuments  (ORDs).^ 

•  The  Direetor,  Command,  Control,  Communieations,  and 
Computers  Systems  Direetorate  (J-6)  (the  Joint  Staff  J-6)  had 
not  updated  the  Joint  Command,  Control,  Communieations, 
Computers,  and  Intelligenee  Program  Assessment  Tool 
(JCPAT)  database  to  inelude  all  interoperability  eertifieations; 
Test  and  Evaluation  Master  Plans  (TEMPs);  and  Command, 
Control,  Communieations,  Computers,  and  Intelligenee  (C4I) 
support  plans  for  DoD  systems. 


‘Title  of  instruction  changed  as  a  result  of  the  update. 

^Two  interoperability  certifications  exist;  the  first  for  requirements  and  supportability  documents  and  the 
second  for  interoperability  certification  testing.  The  Chairman  of  the  Joint  Chiefs  of  Staff  (J-6)  and  the 
Joint  Interoperability  Test  Command  are  responsible  for  requirements  and  supportability  document 
certification  and  for  interoperability  certification  testing,  respectively. 

^The  updated  DoD  Instruction  4630.8  assigned  USJFCOM  with  the  responsibility  to  collect,  consolidate, 
and  prioritize  the  information  technology  and  national  security  systems’  interoperability  and 
supportability  requirements  for  emerging  and  fielded  Joint  Task  Force  systems  using  input  from  Defense 
agencies  and  Military  Departments. 
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•  The  Chairman  of  the  Joint  Chiefs  of  Staff;  the  Under 
Seeretary  of  Defense  for  Aequisition,  Teehnology,  and 
Logisties;  the  ASD(C3I);  and  the  Commander,  U.S.  Joint 
Forees  Command,  in  eoordination  with  the  Direetor, 
Operational  Test  and  Evaluation,  had  not  established  eriteria 
for  the  Defense  ageneies  and  the  Military  Departments  to 
plaee  DoD  systems  on  the  Interoperability  Wateh  List. 

•  The  Direetor,  Operational  Test  and  Evaluation  had  not 
eompleted  a  plan  to  inelude  testing  of  eritieal  operational 
issues  addressing  interoperability  and  information  assuranee 
in  the  Global  Information  Grid. 

Without  eonsistent  guidanee  that  makes  eombat  and  materiel  developers 
analyze  programs  using  an  operational  arehiteeture  view,  the  DoD  is  at 
risk  of  developing  systems  that  operate  independently  of  other  systems 
and  of  not  fully  realizing  the  benefits  of  interoperable  DoD  systems  to 
satisfy  the  needs  of  the  warfighter  as  outlined  in  Joint  Vision  2020. 
However,  the  reeent  DoD  revisions  of  polieies  in  aehieving 
transformation  through  interoperability  may  have  eonsolidated  some 
fragmented  aetivities. 


Interoperability  and  Information  Assurance  Policies 


The  following  provides  an  overview  of  DoD  and  Joint  Staff  polieies  eoneerning 
interoperability  and  information  assuranee.  Appendix  F  provides  a  detailed 
diseussion  of  the  poliey. 

Interoperability  Policy.  The  DoD  and  Joint  Staff  established  poliey  guidanee 
eoneerning  interoperability  during  the  requirements  and  aequisition  proeesses. 

DoD  Policy.  The  poliey  requires  that  joint,  eombined,  eoalition,  and 
interageney  missions  must  be  supported  through  interoperable  information 
teehnology  and  national  seeurity  systems  in  global  operations  aeross  the 
peaee-eonfliet  speetrum.  All  information  teehnology  and  national  seeurity 
systems  for  U.S.  Forees  use  are  to  be  eonsidered  for  use  by  joint,  allied,  and 
other  U.S.  Government  departments  and  ageneies.  Information  teehnology  and 
national  seeurity  systems’  interoperability  and  supportability  requirements  are  to 
be  identified  through  the  mission  area  integrated  arehiteetures.  Interoperability 
requirements  are  required  to  be  managed,  evaluated,  and  reported  throughout 
the  life  of  the  system.  All  DoD  aequisition  eategory  programs  will  use  a  C4I 
Support  Plan  to  doeument  interoperability  requirements.  Additionally, 
interoperability  and  supportability  requirements  are  required  to  be  balaneed  with 
the  need  for  information  assuranee. 

Joint  Staff  Policy.  The  poliey  states  that  interoperability  KPPs  in  an 
ORD  define  the  level  of  interoperability  for  the  proposed  system  and  that  ORDs 
must  be  eertified  before  eaeh  milestone,  regardless  of  aequisition  eategory,  for 
eonformanee  with  joint  national  seeurity  systems  and  interoperability  standards. 
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Failure  to  meet  a  KPP  ean  be  eause  for  the  system  to  be  reevaluated  or  the 
program  to  be  reassessed  or  terminated.  In  addition,  USJFCOM  is  to  provide 
eomments  to  the  Joint  Staff  J-6  on  interoperability  issues  for  all  aequisition 
eategory  programs  to  ensure  that  eaeh  ORD  eontains  information  exehange 
requirements  and  operational  views.  Combatant  eommanders,"^  Military 
Departments,  and  DoD  ageneies  are  to  ineorporate  interoperability  testing  into 
their  overall  testing  plans  in  eoordination  with  DISA  and  the  Joint 
Interoperability  Test  Command  (JITC).  Further,  JITC  is  to  eertify  test  results 
for  all  interoperability  system  tests.  The  Joint  Staff  J-6  issues  an  interoperability 
system  validation  memorandum  based  on  the  testing  reports  from  the  JITC. 

Information  Assurance  Policy.  The  DoD  and  Joint  Staff  established  poliey 
guidanee  eoneerning  information  assuranee  during  the  aequisition  proeess. 

DoD  Policy.  The  poliey  established  the  DoD  Information  Teehnology 
Seeurity  Certifieation  and  Aeereditation  Proeess  (the  Proeess)  for  seeurity 
eertifieation  and  aeereditation  of  unelassified  and  elassified  information 
teehnology.  The  Proeess  sets  forth  the  aetivities  and  management  strueture  to 
eertify  and  aeeredit  information  teehnology  systems  that  will  maintain  the 
seeurity  posture  of  the  Defense  Information  Infrastrueture.  Further,  the 
Direetor,  Operational  Test  and  Evaluation  is  not  to  approve  test  plans  unless 
they  eontain  a  well-defined  strategy  for  addressing  information  assuranee 
eoneerns. 

Joint  Staff  Policy.  The  poliey  states  that  information  assuranee  is 
required  for  all  DoD  systems  that  are  used  to  enter,  proeess,  store,  display,  or 
transmit  DoD  information,  regardless  of  elassifieation  or  sensitivity. 

Information  assuranee  requirements  are  to  be  eodeveloped  and  eoevolved  with 
those  for  information  interoperability.  The  poliey  also  states  that  information 
assuranee  is  eritieal  to  the  military’s  ability  to  eonduet  warfare  and  is  the 
responsibility  of  all  modern  warfighters;  a  risk  assumed  by  one  organization,  at 
any  organization  level,  ean  be  a  risk  imposed  on  all  organizations.  Therefore, 
the  requirement  for  implementing  information  assuranee  applies  at  all  DoD 
organization  levels.  Further,  information  assuranee  for  DoD  information 
systems  and  networks  requires  a  strategy  that  integrates  the  eapabilities  of 
people,  operations,  and  teehnology  to  ensure  survivability  and  mission 
aeeomplishment. 


Evaluation  of  Interoperability  and  Information  Assurance 
Policy 


The  ASD(C3I)  had  interoperability  polieies  that  were  ineonsistent  with  those  of 
the  Chairman  of  the  Joint  Chiefs  of  Staff.  Appendix  F  identifies  the 
interoperability  and  information  assuranee  polieies  that  the  Defense  ageneies  and 
the  Military  Departments  must  implement.  Appendix  G  provides  a  flow  ehart  of 
the  interoperability  requirements  review  proeess  for  DoD  weapon  systems. 


‘'Formerly  commanders-in-chief. 


6 


Inconsistent  Policy.  DoD  had  issued  at  least  six  separate  poliey  doeuments  for 
interoperability  and  at  least  three  separate  poliey  doeuments  for  information 
assuranee  beeause  DoD  dispersed  authority  for  interoperability  and  information 
assuranee  poliey  to  various  organizations  within  the  Offiee  of  the  Seeretary  of 
Defense.  Speeifieally,  direetorates  in  the  offiees  of  the  Under  Seeretary  of 
Defense  for  Aequisition,  Teehnology,  and  Logisties;  the  ASD(C3I);  the 
Direetor,  Operational  Test  and  Evaluation;  and  the  Joint  Staff  had  established 
interoperability  and  information  assuranee  polieies  and  instruetions  that  apply  to 
DoD.  The  DoD  interoperability  and  information  assuranee  polieies  and 
instruetions  deseribe  proeesses  for  reviewing  system  interoperability 
requirements  during  the  requirement  generation  stage  and  require  that  the 
interoperability  of  system  arehiteetures  be  tested  to  ensure  that  information 
teehnology  and  national  seeurity  systems  eommunieate. 

During  the  audit,  ASD(C3I)  updated  DoD  Direetive  4630.5,  “Interoperability 
and  Supportability  of  Information  Teehnology  (IT)  and  National  Seeurity 
Systems  (NSS)”  and  DoD  Instruetion  4630.8  on  January  11  and  May  2,  2002, 
respeetively,  beeause  the  outdated  DoD  Direetive  and  Instruetion  had  not  been 
updated  for  10  years.  Both  updated  polieies  eaneeled  the  previous  versions 
signed  in  1992.  Updates  in  2002  to  the  DoD  Instruetion  resolved  the 
ineonsisteneies  and  synehronized  polieies  and  instruetions  for  the  first  time. 

Information  Assurance  Policy.  The  level  of  information  assuranee  eontrols 
varied  throughout  the  DoD  beeause  no  standard  methodology  existed  to  measure 
the  progress  of  information  assuranee,  and  no  DoD-wide  information  seeurity 
plan  was  implemented  to  proteet  and  defend  the  DoD  systems  and  networks, 
aeeording  to  prior  Inspeetor  General  of  the  Department  of  Defense  reviews. 

The  Inspeetor  General  of  the  Department  of  Defense  in  Report  No.  D-2001-182, 
“Information  Assuranee  Challenges,”  stated  that  until  DoD  implements  a 
eonsolidated  poliey  approaeh  to  information  assuranee  and  fully  ineorporates 
information  assuranee  requirements  into  its  ongoing  arehiteetural  efforts, 
seeurity  polieies  and  proeedures  will  eontinue  to  be  fragmented,  and  DoD 
Components  will  eontinue  to  provide  varied  and  ineonsistent  levels  of 
information  assuranee.  Consolidating  information  assuranee  poliey  would 
eliminate  the  need  to  keep  all  other  DoD  information  assuranee  polieies  updated. 
We  will  review  speeifie  requirements  in  upeoming  audits,  as  diseussed  in 
Appendix  A,  “Other  Matters  of  Interest.” 

Revisions  to  Guidance.  When  the  Chairman  of  the  Joint  Chiefs  of  Staff  revised 
interoperability  polieies,  those  revisions  were  not  immediately  refleeted  in  DoD 
Instruetion  4630.8.  However,  during  this  audit,  DoD  updated  the  polieies  to 
reduee  ineonsistent  attributes  that  would  eonfuse  those  who  will  be  required  to 
implement  interoperability  and  information  assuranee  polieies.  Furthermore, 
the  Offiee  of  the  Seeretary  of  Defense,  in  eonjunetion  with  the  Chairman  of  the 
Joint  Chiefs  of  Staff,  should  implement  a  proeess  that  timely  updates  all 
interoperability  and  information  assuranee  polieies  when  interoperability  or 
information  assuranee  poliey  revisions  oeeur. 
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Interoperability  Requirements 


Public  Law.  Section  922,  Public  Law  105-261,  “Strom  Thurmond  National 
Defense  Authorization  Act  for  Fiscal  Year  1999,”  expressed  the  sense  of 
Congress  concerning  joint  warfighting  experimentation.  In  that  provision. 
Congress  discussed  the  designation  of  a  commander  of  a  combatant  command  to 
have  the  mission  of  joint  warfighting  experimentation,  as  a  key  step  in 
exploiting  advanced  technologies,  new  organizational  structures  and  new  joint 
operational  concepts  to  transform  the  conduct  of  military  operations  by  the  U.S. 
Armed  Forces.  Congress  also  expressed  its  sense  that  such  a  commander  should 
be  provided  with  appropriate  and  sufficient  resources  for  joint  war  fighting 
experimentation  and  listed  the  responsibilities  and  authorities  that  should 
accompany  that  designation,  including  improving  interoperability;  reducing 
unnecessary  redundancy;  synchronizing  technology  fielding;  and  making 
recommendations  to  the  Chairman  of  the  Joint  Chiefs  of  Staff  on  mission  needs 
statements  and  operational  requirements  documents.  Section  485,  title  10, 
United  States  Code,  was  added  by  section  923  of  Public  Law  105-261  to  provide 
for  an  annual  reporting  requirement  from  the  combatant  commander  to  the 
Secretary,  and  then  to  the  Congress,  on  the  joint  experimentation  efforts  for 
each  preceding  fiscal  year. 

Comparison  of  ORDs  to  Joint  Mission  Architectures.  The  DoD  designated 
the  Commander,  USJFCOM  as  the  DoD  Joint  Force  Integrator.^  However,  the 
USJFCOM  did  not  compare  ORDs  to  joint  mission  architectures  before  system 
development  started.  This  condition  occurred  because  the  USJFCOM,  although 
responsible  for  conducting  interoperability  analysis,  was  not  assessing  the 
interoperability  requirements  within  a  joint  mission  architecture  and  b^ecause  it 
did  not  have  the  necessary  analytical  or  quantitative  modeling  and  simulation 
tools.  In  comments  to  the  report,  ASD(C3I)  stated  that  the  Joint  Staff  must 
refine  the  joint  mission  architectures  and  the  subordinate  mission  areas  before 
USJFCOM  can  conduct  an  effective  comparative  analysis  of  operational 
requirements. 

Interoperability  Analysis.  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruction  6212. OIB  requires  the  USJFCOM  to  provide  comments  to  the  Joint 
Staff  J-6  on  interoperability  issues  for  all  acquisition  category  programs  so  that 
each  ORD  contains  interoperability  KPPs  and  information  exchange 
requirements.  However,  the  USJFCOM  did  not  analyze  the  ORD  requirements 
of  a  proposed  DoD  system  within  the  context  of  ORD  requirements  of  other 
systems  with  which  a  proposed  DoD  system  must  interoperate  and  exchange 
information.  Instead  of  analyzing  the  ORDs  that  the  system  will  interoperate 
with,  the  USJFCOM  stated  that  it  reviews  ORDs  within  the  context  of  CRDs 
and  measures  interoperability  and  integration  requirements  against  ORDs  that 


^DoD  Instruction  4630.8  states  that  the  USJFCOM,  in  an  expanded  role  as  the  DoD  Joint  Force 
Integrator,  is  responsible  for  enhancing  interoperability  and  joint,  combined,  and  coalition  capabilities  by 
recommending  changes  in  doctrine,  organizations,  training  and  education,  materiel,  leader  development, 
and  persoimel. 
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relate  to  CRDs  and  against  other  known  DoD  systems  outside  the  CRDs.  The 
updated  DoD  Instruetion  4630.8  assigns  the  USJFCOM  with  the  responsibility 
to  eolleet,  eonsolidate,  and  prioritize  the  information  teehnology  and  national 
seeurity  systems  interoperability  and  supportability  requirements  for  emerging 
and  fielded  Joint  Task  Foree  systems  using  input  from  Defense  ageneies  and 
Military  Departments.  Additionally,  the  Instruetion  requires  that  USJFCOM 
partieipate  in  the  requirements  validation  proeess  for  information  teehnology  and 
national  seeurity  systems  by  reviewing  and  eonfirming  that  the  interoperability 
information  exehange  requirements  and  KPPs  are  suffieient  for  requirements 
doeuments.  This  assessment  is  based  on  the  warfighter’s  perspeetive  using  joint 
mission  area  integrated  arehiteetures. 

Interoperability  Review  Assessment.  To  provide  interoperability 
eomments,  USJFCOM  subjeet-matter  experts  analyze  ORD  requirements  against 
qualitative  eriteria  established  in  Joint  Staff  poliey  after  reeeiving  the  ORD  from 
the  Joint  Staff  J-6.  However,  the  subjeet-matter  experts  do  not  eompare  the 
proposed  system  ORD  against  the  ORDs  of  the  systems  it  must  be  interoperable 
with,  even  if  required,  beeause  they  do  not  have  the  neeessary  analytieal  or 
quantitative  modeling  and  simulation  tools  to  determine  whether  the  ORD  is 
appropriate  for  the  operational  or  systems  arehiteeture.  As  a  result,  the 
subjeet-matter  experts  eonsider  a  system  interoperable  if  the  ORD  eontains 
interoperability  information  exehange  requirements  and  KPPs,  but  they  do  not 
determine  whether  the  proposed  DoD  system’s  requirements  are  interoperable 
with  other  systems  in  their  intended  mission  arehiteetures.  Therefore,  the 
subjeet-matter  experts  eonduet  only  a  stand-alone  qualitative  assessment  of  the 
ORD.  The  updated  DoD  Instruetion  4630.8  requires  that  the  development  of 
mission  area  integrated  arehiteetures  be  eonsistent  with  the  produets  required  by 
the  C4I  Surveillanee  and  Reeonnaissanee  Arehiteeture  Framework.  The 
ASD(C3I)  stated  that  the  updated  DoD  Instruetion  4630.8,  eombined  with 
guidanee  in  the  DoD  Regulation  5000. 2-R,  “Mandatory  Proeedures  for  Major 
Defense  Aequisition  Programs  (MDAPs)  and  Major  Automated  Information 
System  (MAIS)  Aequisition  Programs,”  April  5,  2002,  that  requires  working 
level  integrated  produet  teams  to  develop  ORDs  and  C4I  support  plans,  will 
assist  the  USJFCOM  in  the  ORD  analysis. 

Tools.  Modeling,  simulations,  or  relationship  databases  may  provide 
USJFCOM  with  the  means  to  synehronize  interoperability  requirements  before 
beginning  system  developmental  work.  Matehing  ORD  interoperability  and 
information  assuranee  requirements  at  the  beginning  is  neeessary  for  later 
speeifieations,  standards,  and  interfaees  to  be  written  in  aequisition  doeument 
plans.  Without  eondueting  upfront  interoperability  requirements  analysis  of 
individual  systems  in  mission  arehiteetures,  DoD  may  be  at  greater  risk  of 
developing  systems  that  do  not  eommunieate  with  eaeh  other.  The  Automated 
Commander-in-Chief  Integrated  System  Tool  (the  Tool)  is  a  DoD  initiative 
under  development  that  eould  assist  the  USJFCOM  in  evaluating  interoperability 
requirements  in  proposed  ORDs  during  the  review  proeess.  The  Tool  is  an 
automated  proeess  intended  to  enable  eombat  developers  to  build  testable  and 
measurable  requirements,  develop  joint  information  exehange  requirements, 
identify  operational  arehiteetures,  and  build  system  arehiteetures  based  on 
operational  needs.  The  Tool  eould  be  a  meehanism  to  provide  a  single. 
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integrated,  and  eoherent  view  of  interoperability  requirements  for  the  Joint 
Staff,  the  Military  Departments,  and  the  eombatant  eommands.  Aeeordingly, 
the  USJFCOM  eould  use  the  Tool  or  similar  meehanism  to  review  the  ORD  for 
interoperability  requirements  within  the  eontext  of  mission  arehiteetures. 
However,  all  Defense  ageneies  and  Military  Departments  would  have  to  agree 
to  use  the  Tool  or  a  similar  tool  as  the  standard  format  for  requirements  sent  to 
USJFCOM. 

Tracking  Requirements.  The  Joint  Staff  J-6  did  not  have  an  updated  Joint 
Command,  Control,  Communieations,  Computers,  and  Intelligenee  Program 
Assessment  Tool  (JCPAT)  database  to  maintain  interoperability  eertifieations, 
TEMPs,  C4I  support  plans,  and  approved  ORDs  for  DoD  systems.  Further, 
after  users  were  granted  aeeess  to  the  JCPAT,  they  had  unlimited  read  aeeess  to 
all  information,  whieh  raises  eoneerns  about  the  need  to  know. 

Maintenance  of  the  Database.  In  May  1999,  the  Joint  Staff  established 
the  JCPAT  on  the  Seeret  Internet  Protoeol  Router  Network  for  DoD 
organizations  to  use  for  traeking  and  staffing  eomments  while  reviewing  the 
requirements  during  the  interoperability  eertifieation  proeess,  as  required  in  the 
Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  6212. OIB.  The  Joint  Staff  J-6 
relies  on  the  DISA  to  maintain  up-to-date  documents  that  contain  interoperability 
and  information  assurance  requirements  for  DoD  systems.  The  documents 
maintained  include  ORDs,  CRDs,  C4I  support  plans,  and  TEMPs.  As  a  result, 
the  JCPAT  is  a  historical  database  that  lists  all  DoD  systems  undergoing 
interoperability  requirements  certification  or  that  have  been  certified  for 
interoperability.  Further,  the  JCPAT  contains  comments  made  on  DoD  systems 
about  interoperability  concerns. 

When  searching  for  interoperability  certified  documents  in  the  JCPAT,  a  user 
could  not  be  assured  that  the  document  located  was  the  most  recently  certified 
document.  More  importantly,  the  Defense  agencies  and  Military  Departments 
did  not  forward  approved  documents,  such  as  the  ORD,  CRDs,  C4I  support 
plans,  or  TEMP,  for  inclusion  in  the  JCPAT  database.  The  final  approved 
documents  were  not  consistently  maintained  in  the  JCPAT  because  the  Joint 
Staff  J-6  depends  on  the  Joint  Requirements  Oversight  Council  and  milestone 
decision  authorities  to  execute  the  requirement  for  Stage  Three  in  the 
requirement  certification  process.®  Therefore,  the  DISA  and  the  USJFCOM 
cannot  compare  new  proposed  ORDs  to  approved  ORDs  to  provide  meaningful 
comments  to  the  Joint  Staff  J-6  on  the  proposed  ORDs. 

For  the  JCPAT  to  be  useful,  final  versions  of  the  documents  must  be  submitted 
and  included  in  the  JCPAT.  Further,  outdated  versions  of  documents  should  be 
archived  so  that  users  can  locate  and  obtain  the  most  up-to-date  approved 
documents  for  making  interoperability  determinations  for  DoD  systems. 


®The  Chairman  of  the  Joint  Chiefs  of  Staff  Instruetion  6212. OIB  requires  posting  the  doeument  into  the 
JCPAT  within  15  days  after  the  Joint  Requirements  Oversight  Couneil  or  the  milestone  deeision  authority 
approves  the  mission  need  statement,  CRD,  or  ORD. 
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Access  to  Database.  Appendix  H  identifies  50  DoD  organizations  that 
have  aeeess  to  the  JCPAT.  In  those  50  DoD  organizations,  756  users  were 
given  aeeess  to  the  JCPAT  through  a  seeure  network.  DISA  identified  756  total 
users;  however,  it  eould  not  designate  whether  those  users  were  DoD  or 
eontraetor  personnel.  As  a  result,  eontraetors  have  read-only  aeeess  to  program 
doeumentation  for  whieh  they  may  not  have  a  need  to  know. 


Oversight  of  Significant  Interoperability  Deficiencies 


Interoperability  Watch  List.  DoD  policy  on  the  Interoperability  Watch  List 
(Watch  List)  did  not  define  a  significant  interoperability  deficiency  that  would 
cause  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Defense 
agencies,  and  the  Military  Departments  to  place  a  DoD  system  on  the  Watch 
List. 

Policy.  DoD  Regulation  5000. 2-R  states  that  all  major  DoD  acquisition 
programs,  programs  on  the  Office  of  the  Secretary  of  Defense  Test  and 
Evaluation  Oversight  list,  legacy  systems,  and  all  programs  and  systems  that 
must  interoperate  with  them  are  subject  to  interoperability  evaluations 
throughout  their  life  cycles  to  validate  their  ability  to  support  mission 
accomplishment.  At  their  discretion,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics;  the  ASD(C3I);  the  Director, 
Operational  Test  and  Evaluation;  and  the  Joint  Staff  (the  four  signatories) 
decide,  based  on  the  Interoperability  Senior  Review  Panel’s^  recommendations, 
to  place  programs  and  systems  deemed  to  have  significant  interoperability 
deficiencies  on  the  Watch  List.  DoD  systems  are  to  remain  on  the  Watch  List 
until  program  managers  take  corrective  actions  to  address  identified 
interoperability  deficiencies.  The  Interoperability  Senior  Review  Panel  is 
required  to  prepare  quarterly  reports  summarizing  the  activities  of  DoD  systems 
and  programs  on  the  Watch  List. 

Inclusion  of  DoD  Systems  with  Significant  Interoperability  Deficiencies  on 
the  Watch  List.  Although  the  policy  applies  to  all  programs  and  systems,  as  of 
July  2002,  no  programs  had  been  or  are  on  the  Watch  List.^  Eurther,  the 
Interoperability  Senior  Review  Panel  did  not  prepare  any  quarterly  reports  that 
addressed  DoD  systems  with  interoperability  testing  concerns  during  EY  2001. 


’The  Interoperability  Senior  Review  Panel  is  composed  of  senior  leaders  from  the  Offices  of  the  Under 
Secretary  of  Defense  for  Acquisition  Technology  and  Logistics;  the  ASD(Ch);  the  Joint  Staff;  the 
USJFCOM;  the  Director  for  Programs,  Analysis,  and  Evaluation;  and  the  Director,  Operational  Test  and 
Evaluation. 

®The  Director,  Operational  Test  and  Evaluation  stated  that  the  original  intent  was  for  organizations,  such 
as  the  Military  Communication  and  Electronics  Board,  the  USJECOM,  and  others,  to  identify  DoD 
systems  for  the  Watch  List.  The  Interoperability  Senior  Review  Panel  has  discussed  criteria.  One 
criteria  is  the  overall  impact  the  lack  of  interoperability  has  on  a  mission  area.  The  Interoperability 
Senior  Review  Panel  has  considered  the  U.S.  Air  Eorce  Situational  Awareness  Date  Link  as  a  canddate 
for  placement  on  the  Watch  List;  however,  the  Interoperability  Senior  Review  Panel  has  been  unable  to 
reach  consensus  on  the  placing  the  Situational  Awareness  Date  Link  on  the  Watch  List. 
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However,  the  Direetor,  Operational  Test  and  Evaluation  eited  interoperability 
testing  eoneerns  for  21  DoD  systems  in  the  FY  2001  Annual  Test  Report  to 
Congress,  as  shown  in  the  following  table. 


Programs  with  Interoperability  Deficiencies 

in  the  FY  2001  Annnal  Test  Report 

Number 

Military 

of  DoD 

Department 

Systems 

Army 

8 

Navy 

7 

Air  Foree 

5 

Other  DoD 

1 

Total 

21^ 

The  Direetor,  Operational  Test  and  Evaluation  stated  that  identifying 
interoperability  defieieneies  in  the  FY  2001  Annual  Report  is  a  positive  sign  that 
DoD  polieies  are  working  and  that  program  managers  are  now  obligated  to 
address  the  Direetor,  Operational  Test  and  Evaluation  eoneerns  before 
progressing  with  development  and  produetion.  Furthermore,  the  Direetor, 
Operational  Test  and  Evaluation  stated  that  many  of  the  systems  shown  in  the 
table  will  be  tested  and  validated  over  a  series  of  already  seheduled  test  events 
outlined  in  the  applieable  system’s  TEMP.  However,  some  of  the  programs 
were  not  mature  enough  in  development  to  address  interoperability  issues,  while 
other  systems  were  having  to  “eateh  up”  to  reeent  polieies,  beeause  the  polieies 
were  not  in  existenee  in  the  system’s  early  development  phase. 

DoD  Regulation  5000. 2-R  states  that  DoD  systems  that  have  a  signifieant 
interoperability  defieieney  may  be  added  to  the  Wateh  List.  Beeause  the  phrase 
signifieant  interoperability  defieieney  is  ambiguous,  a  more  eoneise  definition  is 
required  to  determine  when  a  DoD  system  should  be  added  to  the  Wateh  List. 
The  Interoperability  Senior  Review  Panel  eould  use  the  requirements  in  DoD 
Regulation  5000. 2-R  on  interoperability  and  information  assuranee  as 


^Of  the  21  programs,  9  are  Acquisition  Category  I,  8  are  Acquisition  Category  II,  and  4  are  Acquisition 
Category  III.  Those  programs  are  on  the  Director,  Operational  Test  and  Evaluation’s  Oversight  Watch 
List.  The  Director,  Operational  Test  and  Evaluation  stated  that,  because  those  programs  were  on  the 
Oversight  Watch  List,  many  of  the  system  level  issues  would  be  resolved  at  the  milestone  decision 
reviews.  However,  he  stated  that  the  concern  was  with  the  systems  not  on  the  Oversight  Watch  List, 
such  as  Acquisition  Category  III  and  IV  systems,  transitioning  advanced  concept  technology 
demonstrations,  and  fielded  legacy  systems. 
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criteria  for  determining  inelusion  of  systems  on  the  Wateh  List.  DoD 
Regulation  5000. 2-R  requires  that  DoD  system  program  offiees  have: 

•  an  interoperability  eertifieation  from  the  Joint  Staff  J-6, 

•  approval  of  the  C4I  support  plan  and  information  assuranee  strategy 
from  the  ehief  information  offieer, 

•  verifieation  that  the  program  manager  used  the  Joint  Teehnieal 
Arehiteeture  standards  and  guidelines  in  system  development,  and 

•  an  approved  Systems  Seeurity  Authorization  Agreement. 

By  aetively  using  the  Wateh  List,  the  DoD  would  have  a  method  of  ensuring 
eomplianee  with  the  interoperability  and  information  assuranee  based  on  DoD 
Regulation  5000. 2-R  requirements.  The  updated  DoD  Instruetion  4630.8 
reinforees  the  establishment  of  the  Wateh  List.‘°  However,  the  revised 
instruetion  does  not  state  speeifie  eriteria  for  systems  to  be  ineluded  on  the 
Wateh  List.  The  phrase  “eritieal  for  mission  effeetiveness”  allows  a  broad 
interpretation. 

Measuring  Progress.  DoD  Regulation  5000. 2-R  states  that  when  a  DoD 
system  is  plaeed  on  the  Wateh  List,  the  program  offiee  is  to  provide  periodie 
updates  of  eurrent  status  towards  eorreeting  identified  defieieneies  to  the 
Interoperability  Senior  Review  Panel.  The  program  manager,  or  other 
eognizant  offieial,  and  the  responsible  test  organization,  in  eonjunetion  with 
JITC,  provide  those  updates.  Those  updates  support  an  assessment  as  to 
whether  interoperability  issues  are  being  adequately  addressed,  and  whether  a 
status  ehange  is  warranted;  speeifieally,  whether  the  program  or  system  should 
be  removed  from  the  Wateh  List,  kept  on  the  Wateh  List,  or  proposed  for  test 
and  evaluation  oversight.  Staff  members  of  the  Interoperability  Senior  Review 
Panel  are  to  eonduet  quarterly  reviews  to  determine  the  program  manager’s 
progress  towards  addressing  identified  interoperability  defieieneies. 

To  strengthen  the  review  proeess  for  measuring  eorreetive  aetions  to  mitigate 
the  defieieneies,  program  offiee  eorreetive  aetions  taken  on  Aequisition 
Category  I  programs  and  less  than  Aequisition  Category  I  programs  on  the 
Wateh  List  should  be  diseussed  in  the  Defense  Aequisition  Exeeutive  Summary 
reporting  proeess  and  during  program  reviews,  respeetively. 

Defense  Acquisition  Executive  Summary.  DoD  Regulation  5000. 2-R 
states  that  the  Defense  Aequisition  Exeeutive  Summary  is  a  multi-part 
doeument,  whieh  reports  program  information  and  assessments;  program 


‘°DoD  Instruction  4630.8  requires  that  the  Director,  Operational  Test  and  Evaluation  establish,  in 
conjunction  with  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics;  the  DoD 
Chief  Information  Officer;  the  Chairman  of  the  Joint  Chiefs  of  Staff;  and  the  USJFCOM,  an 
Interoperability  Watch  List  to  provide  DoD  oversight  for  those  systems  that  interoperability  is  deemed 
critical  to  mission  effectiveness,  but  is  not  being  adequately  addressed.  Systems  considered  for  the 
Interoperability  Watch  List  may  be  preacquisition  systems,  acquisition  programs  (any  acquisition 
category),  already  fielded  systems,  or  combatant  command-unique  procurements. 
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manager,  program  executive  officer,  and  Component  acquisition  executive 
comments;  and  cost  and  funding  data  primarily  for  Acquisition  Category  I 
programs.  The  Defense  Acquisition  Executive  Summary  is  an  early  warning 
report  to  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  and  the  ASD(C3I),  which  describes  actual  program  problems,  warns 
of  potential  program  problems,  and  describes  mitigating  actions  taken.  At  a 
minimum,  the  Defense  Acquisition  Executive  Summary  reports  program 
assessments,  including  interoperability,  unit  costs,  and  current  estimates,  and 
the  status  of  exit  criteria  and  vulnerability  assessments.  The  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics;  the  Under  Secretary  of 
Defense  (Comptroller);  Director,  Operational  Test  and  Evaluation;  the  Cost 
Analysis  Improvement  Group  in  the  Office  of  the  Secretary  of  Defense;  the 
Component  acquisition  executives;  and  the  program  executive  officers  review  or 
assess  the  Defense  Acquisition  Executive  Summary  reports.  During  those 
reviews  or  assessments,  we  believe  those  groups  should  determine  whether  the 
program  has: 

•  shown  significant  interoperability  deficiencies, 

•  been  placed  on  the  Watch  List,  and 

•  made  progress  towards  correcting  the  significant  interoperability 
deficiencies  to  be  removed  from  the  Watch  List. 

By  reviewing  significant  interoperability  deficiencies  during  the  review  process, 
the  Defense  Acquisition  Executive  Summary  review  groups  may  have  focused 
additional  attention  on  the  nine  Acquisition  Category  I  programs  that  have 
interoperability  testing  concerns  in  the  EY  2001  Annual  Test  Report  to 
Congress. 

Program  Reviews.  At  milestone  and  appropriate  program  reviews,  the 
milestone  decision  authority  should  be  focusing  additional  attention  on  programs 
that  are  on  the  Watch  List  to  determine  whether  the  program  offices  are  making 
satisfactory  progress  towards  correcting  significant  interoperability  deficiencies. 

Testing  Interoperability  and  Information  Assurance 
Requirements  in  the  Global  Information  Grid 

On  March  31,  2000,  the  Deputy  Secretary  of  Defense  directed  the  Director, 
Operational  Test  and  Evaluation  to  include  critical  operational  issues  addressing 
interoperability  and  information  assurance  in  the  GIG  operational  test  and 
evaluation. 

The  GIG  is  not  one  system,  but  an  end-to-end  set  of  information  capabilities, 
associated  processes,  and  personnel  for  collecting,  processing,  storing, 
disseminating,  and  managing  information  on  demand  to  warfighters,  policy 
makers,  and  support  personnel.  The  GIG  includes  all  owned  and  leased 
communication  and  computing  systems,  services,  software,  data,  security 
services,  national  security  systems,  and  associated  services  necessary  to  achieve 
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Information  Superiority.  In  this  regard,  the  Direetor,  Operational  Test  and 
Evaluation  plans  to  eventually  ereate  a  joint  network  to  simulate  the  GIG 
eonsisting  of  equipment  at  Military  Department  test  faeilities.“  That  simulated 
network  would  show  how  DoD  systems  will  eommunieate  in  a  joint  and  allied 
environment.  However,  the  Direetor,  Operational  Test  and  Evaluation  had  not 
formalized  a  plan  that  diseusses  the  required  resourees  needed  to  eonstruet  the 
network. 

Military  Departments,  system  designers,  and  system  testers  need  to  eoordinate 
their  resourees  to  adequately  test  interoperability  and  information  assuranee 
among  DoD  systems  in  the  GIG  and  determine  their  mission  eapability  in  a  joint 
environment.  System  testers  design  operational  tests  around  eritieal  operational 
issues  that  are  derived  from  operational  requirements  in  the  ORD.  However, 
the  GIG  is  based  on  many  DoD  systems  that  work  together  as  an  arehiteeture  of 
a  mission  area.  Therefore,  testing  of  interoperability  and  information  assuranee 
in  individual  DoD  systems  that  are  part  of  the  GIG  must  be  ineluded  as  eritieal 
operational  issues  so  that  testers  will  fully  test  the  operational  effeetiveness  and 
suitability  of  the  individual  systems  as  they  operate  in  the  GIG.  The  Direetor, 
Operational  Test  and  Evaluation  stated  that  the  ehallenge  is  to  identify  the 
appropriate  arehiteeture  and  the  ability  to  exeeute  a  test  that  takes  into  aeeount 
the  seope  of  that  arehiteeture. 

Conclusion 

Without  eonsistent  guidanee  that  makes  eombat  and  materiel  developers  analyze 
programs  using  an  operational  arehiteeture  view,  the  DoD  is  at  risk  of 
developing  systems  that  operate  independently  of  other  systems  and  of  not  fully 
realizing  the  benefits  of  interoperable  DoD  systems  to  satisfy  the  needs  of  the 
warfighter  as  outlined  in  Joint  Vision  2020.  DoD  interoperability  and 
information  assuranee  requirements  must  beeome  more  streamlined  and  easier  to 
implement.  Eurther,  information  assuranee  must  be  implemented  appropriately 
and  eommensurate  to  the  level  of  interoperability  required  to  avoid  fielding  DoD 
systems  that  do  not  ensure  the  availability,  integrity,  authentieity, 
eonfidentiality,  and  nonrepudiation  of  information.  To  aehieve  this  objeetive, 
eombat  developers'^  must  have  the  eapability  to  eonduet  analyses  of  operational 
requirements  at  the  beginning  of  a  system  to  determine  the  interoperability  and 
information  assuranee  effeets  of  new  and  legaey  systems  on  mission  area 
arehiteetures.'^  Without  this  eapability,  the  eombat  developers  eannot  determine 
whieh  systems  need  to  operate  together  and  whether  those  systems  will  be 
suitable  for  the  warfighter. 


“DoD  has  funded  the  development  of  the  Joint  Distributed  Engineering  Plant  to  coordinate  resources  for 
testing.  The  Director,  Operational  Test  and  Evaluation  stated  that,  as  the  Joint  Distributed  Engineering 
Plant  matures,  he  will  use  it  for  integration  testing  to  provide  that  larger  environment  represented  by  the 
GIG. 

“A  combat  developer  is  the  command  or  agency  that  formulates  doctrine,  concepts,  organization 
makeup,  materiel  requirements,  and  objectives.  Generically,  it  represents  the  user  community  role  in  the 
materiel  acquisition  process. 

‘^The  ASD(Ch),  the  Director,  Operational  Test  and  Evaluation,  and  the  USJECOM  stated  that  the 
Chairman  of  the  Joint  Chiefs  of  Staff  completing  joint  mission  area  architectures  is  critical  to  achieving 
interoperability. 
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Management  Comments  on  the  Finding  and  Audit  Response 

Summaries  of  management  comments  on  the  finding  and  audit  responses  are  in 
Appendix  1. 

Recommendations,  Management  Comments,  and  Audit 
Response 


Redirected  and  Revised  Recommendations.  In  response  to  most  of  the 
recommendations,  we  received  varied  management  comments  to  the  same 
recommendation,  often  without  a  coordinated  corrective  action  plan.  Therefore, 
to  collectively  resolve  the  recommendations  and  to  obtain  succinct  and  attainable 
corrective  action,  we  are  redirecting  Recommendations  1.,  2.,  3. a.,  3.b.,  4. a., 
and  4.b.  to  the  Co-Chair,  Interoperability  Senior  Review  Panel,  which  consists 
of  senior  leaders  from  the  Offices  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics;  the  ASD(C3I);  the  Joint  Staff;  the 
USJFCOM;  the  Director  for  Programs,  Analysis,  and  Evaluation;  and  the 
Director,  Operational  Test  and  Evaluation.  Further,  in  response  to  comments 
by  the  Director,  Interoperability,  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics;  the  Inspector  General,  USJFCOM;  and 
the  Director,  Joint  Staff,  we  revised  Recommendations  1.,  3. a.,  and  4.b., 
respectively,  so  that; 

•  the  revision  process  for  interoperability  and  information  assurance 
policies  will  be  jointly  implemented  instead  of  coordinated  among  the 
applicable  organizations; 

•  the  Commander,  USJFCOM  will  compare  the  ORDs  of  proposed 
DoD  systems  with  joint  mission  area  architectures,  as  they  are 
completed,  against  the  ORDs  in  the  related  joint  mission  area 
architecture  to  verify  the  completeness  of  stated  interoperability 
requirements;  and 

•  JCPAT  access  will  be  limited  to  users  who  have  a  need  to  know. 

1.  We  recommend  that  the  Co-Chair,  Interoperability  Senior  Review 
Panel,  in  concert  with  the  applicable  membership  of  the  Interoperability 
Senior  Review  Panel,  implement  a  process  that  timely  integrates  revisions 
for  interoperability  and  information  assnrance  policies  into  the  applicable 
DoD  and  Chairman  of  the  Joint  Chiefs  of  Staff  interoperability  and 
information  assnrance  policies. 

Director,  Architectnre  and  Interoperability,  Office  of  the  ASD(C3I), 
Comments.  The  Director  concurred,  stating  that  his  office  will  work  with  the 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  and  the  Joint  Staff  to  implement  a  process  for  timely  revision  and 
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synchronization  of  DoD  interoperability  and  information  assuranee  polieies. 
Further,  the  Direetor  stated  that  the  Interoperability  Senior  Review  Panel  will  be 
used  to: 

•  eoordinate  the  annual  review  and  synehronization  and  review,  if 
required,  for  the  following  polieies  and  implementing  instructions 
that  affect  interoperability:  DoD  Direetive  4630.5;  DoD 
Instruetion  4630.8;  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruetion  3170. OIB,  “Requirements  Generation  System,”  April  15, 
2001;  and  Chairman  of  the  Joint  Chiefs  of  Staff 

Instruetion  62 12. OIB;  and 

•  ensure  eonsisteney  of  those  polieies  and  implementing  instruetions 
with  the  5000  series  poliey  doeuments  for  aequisition;  DoD 
Instruetion  5200.40,  “DoD  Information  Teehnology  Seeurity 
Certifieation  and  Aeereditation  Proeess  (DITSCAP),”  Deeember  30, 
1997;  and  Chairman  of  the  Joint  Chiefs  of  Staff 

Instruetion  6510.01C,  “Information  Assuranee  and  Computer 
Network  Defense,”  May  1,  2001. 

For  the  eomplete  text  of  the  Direetor ’s  eomments,  see  the  Management 
Comments  seetion  of  the  report. 

Director,  Joint  Staff  Comments.  The  Direetor  eoneurred,  subjeet  to  the 
inelusion  of  his  eomments,  and  suggested  adding  the  following  to  the 
reeommendation : 

We  also  recommend  the  implementation  of  a  process  which  addresses, 
funds,  and  implements  aspects  of  joint  battle  management  command 
and  control  interoperability  and  coimectively;  validation  and/or 
allocation  of  resources  to  acquire  Joint  systems  and  create  offices 
supporting  integration  of  materiel  and  non-materiel  solutions  for 
broad  mission  capability  areas. 

The  Director  stated  that  he  made  the  suggestion  beeause  the  report  did  not 
address  the  issue  that  DoD  was  not  effeetively  struetured  to  organize,  train,  and 
equip  joint  eapabilities.  Further,  he  stated  that  a  joint  proeess  was  not 
established  to  delineate  who  is  responsible  and  aeeountable  for  developing  and 
aequiring  joint  eommand  and  eontrol  systems  and  integrating  eapabilities.  For 
the  eomplete  text  of  the  Direetor ’s  eomments,  see  the  Management  Comments 
seetion  of  the  report. 

Director,  Interoperability,  Office  of  the  Under  Secretary  of  Defense  for 
Acqnisition,  Technology,  and  Logistics  Comments.  The  Direetor  neither 
eoneurred  nor  noneoneurred;  however,  he  stated  that  the  wording  “in 
eoordination  with”  eould  be  eonstrued  to  give  the  ASD(C3I)  primaey  over  the 
Under  Seeretary  of  Defense  for  Aequisition,  Teehnology,  and  Logisties  and  the 
Joint  Staff  in  integrating  revisions  to  all  doeuments  related  to  interoperability. 
Further,  he  stated  that  eaeh  of  those  organizations  is  responsible  for  one  or  more 
of  those  doeuments.  Therefore,  the  Direetor  reeommended  that  the  phase  “in 
eoordination  with”  be  ehanged  to  “jointly  with”  or  that  the  first  four  lines  be 
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rewritten  to  read,  “We  reeommend  that  the  Assistant  Seeretary  of  Defense 
(Command,  Control,  Communieations,  and  Intelligenee);  the  Under  Seeretary  of 
Defense  for  Aequisition,  Teehnology,  and  Logisties;  and  the  Chairman  of  the 
Joint  Chiefs  of  Staff  jointly  implement. ...”  For  the  eomplete  text  of  the 
Direetor’s  eomments,  see  the  Management  Comments  seetion  of  the  report. 

Co-Chair,  Interoperability  Senior  Review  Panel  Comments.  The  Co-Chair 
agreed,  stating  that  the  Interoperability  Senior  Review  Panel  will  work  with  the 
Joint  Staff  to  implement  a  proeess  for  timely  revision  and  synehronization  of 
DoD  polieies  regarding  interoperability  and  information  assuranee.  In  addition, 
he  provided  comments  similar  to  those  made  by  the  Director,  Architecture  and 
Interoperability,  Office  of  the  ASD(C3I).  For  the  complete  text  of  the 
Co-Chair’s  comments,  see  the  Management  Comments  section  of  the  report. 

Audit  Response.  The  comments  by  the  Director,  Architecture  and 
Interoperability,  Office  of  the  ASD(C3I);  and  the  Director,  Interoperability, 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  were  responsive,  hi  response  to  the  Director,  Interoperability,  we 
revised  the  recommendation  as  suggested.  Further,  the  suggestion  by  the 
Director,  Joint  Staff  concerning  the  issue  of  organizing,  training,  and  equipping 
joint  capabilities  was  outside  the  audit  scope. 

Because  the  respondents  have  not  yet  established  a  process  for  timely  revision 
and  synchronization  of  DoD  policies  regarding  interoperability  and  information 
assurance,  we  redirected  the  recommendation  to  the  Co-Chair,  Interoperability 
Senior  Review  Panel,  whose  membership  consists  of  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics;  the  ASD(C3I);  the  Joint 
Staff;  the  USJFCOM;  the  Director  for  Programs,  Analysis,  and  Evaluation;  and 
the  Director,  Operational  Test  and  Evaluation,  and  request  that  the  Co-Chair 
provide  additional  comments  on  how  and  when  the  Interoperability  Senior 
Review  Panel  will  establish  a  timely  revision  and  synchronization  process. 

2.  We  recommend  that  the  Co-Chair,  Interoperability  Senior  Review 
Panel,  in  concert  with  the  applicable  membership  of  the  Interoperability 
Senior  Review  Panel,  define  the  term  significant  interoperability  deficiency, 
and  establish  criteria  and  procedures  for  placing  a  DoD  system  with  a 
significant  interoperability  deficiency  on  the  Interoperability  Watch  List. 

Director,  Joint  Staff  Comments.  The  Director  concurred,  subject  to  the 
inclusion  of  his  comments,  and  suggested  adding  the  following  subparagraphs  to 
the  recommendation; 

a.  Develop  Joint  Mission  Area  Architectures  based  on  currently 
available  DoD  assets  and  based  on  those  assets  needed  to  provide 
the  capabilities  for  the  future. 

b.  Provide  an  analysis  branch  within  each  JMA  [Joint  Mission  Area] 
to  assess  Requirement  Document  relevance  against  known 
architecture  needs. 
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c.  Develop  compliant,  certified,  and  standard  analysis  and  predictive 
tools,  such  as  models  or  simulations,  to  assist  in  verifying  the 
completeness  of  stated  interoperability  requirements  in  mission 
area  architectures. 

Director,  Architecture  and  Interoperability,  Office  of  the  ASD(C3I), 
Comments.  The  Director  neither  concurred  nor  nonconcurred;  however,  he 
stated  that  his  office  is  working  with  the  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics;  the  Director,  Operational 
Test  and  Evaluation;  and  the  Joint  Staff  to  develop  procedures  for  addressing 
interoperability  deficiencies  within  existing  processes.  Further,  he  stated  that 
his  office  will  continue  to  work  with  the  Interoperability  Senior  Review  Panel  to 
refine  the  process  for  identifying  programs  with  interoperability  deficiencies  and 
for  nominating  those  programs  as  candidates  for  the  Interoperability  Watch  List. 
The  Director  stated  that,  because  the  Interoperability  Watch  List  is  a  relatively 
new  oversight  tool,  the  development  of  appropriate  criteria  and  procedures  for 
placing  programs  on  the  Interoperability  Watch  List  is  still  in  the  formative 
stages. 

In  addition,  the  Director  stated  that  the  Air  Force’s  Situational  Awareness  Data 
Link  program  was  pursued  as  an  initial  pilot  case  for  nomination  to  the 
Interoperability  Watch  List.  He  stated  that  the  lessons  learned  from  that  pilot 
case  will  be  applied  as  other  programs  are  explored  for  Interoperability  Watch 
List  consideration. 

Director,  Operational  Test  and  Evaluation  Comments.  The  Director 
partially  concurred,  stating  that,  as  a  member  of  the  Interoperability  Senior 
Review  Panel,  his  office  continues  to  work  with  the  other  members  on  refining 
the  Interoperability  Watch  List  processes.  Further,  he  stated  that  the 
Interoperability  Watch  List  is  one  of  many  tools  available  within  DoD  to  address 
interoperability  problems.  The  Director  also  stated  that  the  Interoperability 
Watch  List  policy  states  that  the  list  will  be  used  for  those  systems  with 
interoperability  issues  that  are  not  being  adequately  addressed  by  other  forums. 

The  Director  stated  that  the  Interoperability  Senior  Review  Panel  has  served  a 
valuable  role  over  the  last  18  months  by  coordinating  interoperability  policies 
and  actions  across  DoD  and  by  working  within  existing  forums  to  solve  issues. 
For  example,  the  Interoperability  Senior  Review  Panel  worked  with  the  Office 
of  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  and  Concepts, 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  to  ensure  that  interoperability  is  stressed  within  advanced  concept 
technology  demonstrations  to  avoid  problems  in  the  transition  of  these 
experimental  items  to  the  force. 

In  addition,  the  Director  stated  that  the  criteria  for  Interoperability  Watch  List 
candidates  suggested  by  the  report  were  certainly  a  start  and  that  his  office  and 
the  Interoperability  Senior  Review  Panel  would  use  those  criteria  to  evaluate 
future  candidates.  He  also  stated  that  the  report  cited  the  Director,  Operational 
Test  and  Evaluation’s  Annual  Report  to  Congress  as  a  source  of  systems  for  the 
Interoperability  Watch  List. 


19 


In  conclusion,  the  Director  stated  that,  although  he  believed  that  those  systems 
in  the  Annual  Report  already  reeeived  adequate  attention  through  test  and 
evaluation  oversight  and  other  aequisition  review  proeesses,  he  would  bring 
programs  under  test  and  evaluation  oversight  that  were  not  adequately 
addressing  interoperability  to  the  attention  of  the  Interoperability  Senior  Review 
Panel  for  its  eonsideration. 

Co-Chair,  Interoperability  Senior  Review  Panel  Comments.  The  Co-Chair 
stated  that  the  Interoperability  Senior  Review  Panel  will  eontinue  to  refine  the 
proeess  and  proeedures  to  implement  the  Interoperability  Wateh  List.  Further, 
he  stated  that  the  Interoperability  Senior  Review  Panel’s  pilot  ease  in  2001  was 
the  Air  Foree’s  Situational  Awareness  Data  Link  program.  The  Co-ehair  stated 
that  the  Interoperability  Senior  Review  Panel  determined  that  the  Situational 
Awareness  Data  Link  had  signifieant  interoperability  issues.  In  addition,  he 
stated  that  the  visibility,  whieh  the  Interoperability  Senior  Review  Panel  plaeed 
on  the  Situational  Awareness  Data  Link,  eaused  the  Air  Foree  to  design  an 
interoperable  solution  for  its  elose  air  support  aireraft. 

The  Co-Chair  stated  that  the  Interoperability  Senior  Review  Panel  eontinues  to 
investigate  other  programs  for  the  Interoperability  Wateh  List  and  will  work  to 
find  solutions  within  established  proeesses  of  the  Offiees  of  the  Joint  Staff;  the 
Under  Seeretary  of  Defense  for  Aequisition,  Teehnology,  and  Logisties;  the 
ASD(C3I);  and  the  Direetor,  Operational  Test  and  Evaluation  before  plaeing  a 
system  on  the  Interoperability  Wateh  List. 

Audit  Response.  The  eomments  of  the  Direetor,  Joint  Staff;  the  Direetor, 
Arehiteeture  and  Interoperability,  Offiee  of  the  ASD(C3I);  and  the  Direetor, 
Operational  Test  and  Evaluation  were  responsive.  However,  the  suggestion  that 
the  Direetor,  Joint  Staff  made  eoneerning  the  subparagraphs  to  the 
reeommendation  should  be  proposed  to  the  Interoperability  Senior  Review  Panel 
for  defining  eriteria,  proeesses,  and  proeedures  for  the  Interoperability  Wateh 
List. 

Even  though  the  Direetor,  Interoperability,  Offiee  of  the  Under  Seeretary  of 
Defense  for  Aequisition,  Teehnology,  and  Logisties  and  the  Inspeetor  General, 
USJFCOM  did  not  comment  on  the  reeommendation,  the  comments  by  the 
Co-Chair,  Interoperability  Senior  Review  Panel  satisfied  the  requirement  to 
eomment  beeause  those  offiees  are  represented  on  the  Interoperability  Senior 
Review  Panel. 

In  addition  to  investigating  other  programs  for  the  Interoperability  Wateh  List 
and  working  to  find  solutions  within  established  proeesses  before  plaeing  a 
system  on  the  Interoperability  Wateh  List,  the  Interoperability  Senior  Review 
Panel  should  have  a  elearly  defined  proeess  and  eriteria  for  the  Interoperability 
Wateh  List  to  work  effeetively.  The  speeifie  eriteria,  proeesses,  and  proeedures 
should  be  ineluded  in  DoD  instruetions  so  that  eombat  and  materiel  developers 
understand  the  ramifieations  of  their  systems  not  being  interoperable. 

Therefore,  we  redireeted  the  reeommendation  to  the  Co-Chair,  Interoperability 
Senior  Review  Panel,  and  request  that  he  provide  additional  eomments  on  how 
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and  when  the  Interoperability  Senior  Review  Panel  will  establish  a  elearly 
defined  proeess  and  eriteria  for  plaeing  DoD  weapon  systems  in  the 
development  phase  of  the  aequisition  proeess  on  the  Interoperability  Wateh  List. 

3.  We  recommend  that  the  Co-Chair,  Interoperability  Senior  Review 
Panel,  in  concert  with  the  applicable  membership  of  the  Interoperability 
Senior  Review  Panel,  ensnre  that  the  Commander,  U.S.  Joint  Forces 
Command,  pnrsnant  to  DoD  Instrnction  4630.8,  “Procednres  for 
Interoperability  and  Snpportability  of  Information  Technology  (IT)  and 
National  Secnrity  Systems  (NSS),”  May  2,  2002,  and  the  Chairman  of  the 
Joint  Chiefs  of  Staff  Instrnction  6212.01B,  “Interoperability  and 
Snpportability  of  National  Secnrity  Systems,  and  Information  Technology 
Systems,”  May  8,  2000,  as  part  of  the  operational  reqnirements  docnment 
interoperability  certification  process: 

a.  Compares  the  operational  reqnirements  docnments  of  proposed 
DoD  systems  with  joint  mission  area  architectnres,  as  they  are  completed, 
against  the  other  operational  reqnirements  docnments  in  the  related  joint 
mission  area  architectnre  to  verify  the  completeness  of  stated 
interoperability  reqnirements. 

Inspector  General,  USJFCOM  Comments.  The  Inspector  General  neither 
concurred  nor  nonconcurred;  however,  he  stated  that  die  recommendation  is  not 
achievable  without  the  Joint  Staffs  completion  of  the  joint  mission  area 
architectures.  Further,  he  stated  that  USJFCOM  recognizes  that  the 
development  of  the  joint  mission  area  architectures  is  critical  when  comparing 
requirements  across  programs  and  will  continue  to  pursue  this  need  separately 
and  collectively  in  all  available  forums.  In  addition,  the  Inspector  General 
recommended  that  the  recommendation  be  changed  for  accuracy  to  read; 

Compare  the  operational  requirements  documents  of  proposed  DoD 
systems  with  JMA  [Joint  Mission  Area]  architectures,  as  they  are 
completed,  against  the  other  operational  requirements  documents  in 
the  related  joint  mission  area  architecture  to  verify  the  completeness 
of  stated  interoperability  requirements. 

The  Inspector  General  suggested  this  new  recommendation  because  USJFCOM 
was  not  resourced  to  execute  Recommendations  3. a.  and  3.b. 

Director,  Joint  Staff  Comments.  The  Director  stated  that  the  USJFCOM  was 
required  to  review  only  operational  requirements  documents  as  standalone 
documents  for  compliance  with  capstone  requirements  documents.  Further,  he 
stated  that  USJFCOM  does  not  have  the  authority  or  resources  to  provide  a 
more  in-depth  review  of  operational  requirements  documents  for  interoperability 
within  the  respective  family  of  systems.  Further,  the  Director  suggested  that  the 
recommendation  be  changed  for  accuracy  to  read; 

Compare  all  relevant  requirements  documents  of  proposed  DoD  IT 
[Information  Technology]  and  NSS  [National  Security  System]  (to 
include  SAP  [Special  Access  Program])  systems  with  JMA  [Joint 
Mission  Area]  architectures,  as  they  are  completed,  against  all  other 
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operational  requirements  documents  in  the  related  joint  mission  area 
architecture  to  verify  the  completeness  of  stated  interoperability 
requirements. 

Director,  Architecture  and  Interoperability,  Office  of  the  ASD(C3I), 
Comments.  The  Director  agreed  with  the  underlying  premise  of  the 
recommendation.  However,  the  Director  believes  that  the  recommendation 
should  have  indicated  that,  before  the  Joint  Forces  Command  can  conduct  an 
effective  comparative  analysis  of  operational  requirements: 

•  the  Joint  Staff  must  first  define  the  joint  mission  areas  and 
subordinate  supporting  mission  areas  and 

•  the  DoD  Components  must  develop  mission  area  integrated 
architectures. 

Further,  the  Director  stated  that  DoD  Instruction  4630.8  assigns  to  the  Joint 
Staff  the  responsibility  to: 

Ensure  mission  area  integrated  architectures,  strategies,  concepts,  and 
visions  of  the  DoD  Components  are  synchronized  to  support  IT 
[Information  Technology]  and  NSS  [National  Security  System] 
interoperability  requirements  and  identify  opportunities  for,  and 
impediments  to,  interoperability. 

The  Director  also  stated  that,  once  joint  mission  and  subordinate  mission  areas 
are  defined,  the  Joint  Staff  must  coordinate  with  the  DoD  Components  to  ensure 
that  DoD  develops  a  consistent  and  integrated  set  of  mission  area  integrated 
architectures  across  the  Military  Departments  and  Defense  agencies.  He 
proposed  that  a  recommendation  be  made  in  the  report  for  the  Joint  Staff  to 
further  refine  existing  joint  mission  areas  and  to  define  applicable  subordinate 
mission  areas  to  serve  as  a  basis  for  the  DoD  Components  to  develop  mission 
area  integrated  architectures. 

Co-Chair,  Interoperability  Senior  Review  Panel  Comments.  The  Co-Chair 
agreed  with  the  concept  of  the  recommendation,  stating  that  the  Interoperability 
Senior  Review  Panel  recognizes  that  the  development  of  the  joint  mission  area 
architectures  is  critical  to  the  comparison  of  requirements  across  programs. 
Further,  he  stated  that  Interoperability  Senior  Review  Panel  will  continue  to 
emphasize  this  need  separately  and  collectively  in  all  available  forums. 

However,  the  Co-Chair  stated  that  the  recommendation,  as  drafted,  is  not 
actionable  by  the  Joint  Forces  Command  without  Joint  Staff  completion  of  the 
joint  mission  area  architectures. 

Audit  Response.  The  Inspector  General,  USJFCOM  comments  were 
responsive  to  the  intent  of  our  recommendation.  In  response  to  those 
comments,  we  revised  the  recommendation  as  suggested  by  the  Inspector 
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General,  USJFCOM.  Further,  we  redireeted  the  reeommendation  to  the 
Co-Chair,  Interoperability  Senior  Review  Panel,  and  request  that  he  provide 
additional  eomments  on: 

•  the  interim  proeess  that  the  Interoperability  Senior  Review  Panel  and 
USJFCOM  will  employ  to  eompare  the  ORDs  of  proposed  DoD 
systems  against  the  other  ORDs  in  the  related  mission  area 
arehiteeture  to  verify  the  eompleteness  of  stated  interoperability 
requirements  until  the  joint  mission  area  arehiteetures  are  eompleted; 
and 

•  the  resourees  that  the  Interoperability  Senior  Review  Panel  and 
USJFCOM  require  to  implement  the  reeommendation. 

In  response  to  the  Direetor,  Joint  Staff  eomments  that  USJFCOM  does  not  have 
the  authority  or  resourees  to  provide  a  more  in-depth  review  of  operational 
requirements  doeuments.  Congress  expressed  its  sense  in  Seetion  922,  Publie 
Law  105-261,  that  the  eommander  of  a  eombatant  eommand  should  be  provided 
with  appropriate  and  suffieient  resourees  for  joint  warfighting  experimentation. 

It  listed  the  responsibilities  and  authorities  that  should  aeeompany  that 
designation,  ineluding  improving  interoperability  and  making  reeommendations 
to  the  Chairman  of  the  Joint  Chiefs  of  Staff  on  mission  needs  statements  and 
ORDs.  Aeeordingly,  if  USJFCOM  believes  that  resourees  are  insuffieient,  it 
should  request  the  resourees  needed  to  fulfill  the  eongressional  intent. 

b.  Uses  quantification  analysis  and  predictive  tools,  such  as  models 
or  simulations,  to  assist  in  verifying  the  completeness  of  stated 
interoperability  requirements  in  mission  area  architectures. 

Inspector  General,  USJFCOM  Comments.  The  Inspeetor  General  neither 
eoneurred  nor  noneoneurred;  however,  he  stated  that  Ae  reeommendation  was 
not  fully  aehievable  at  this  time.  Further,  he  stated  that,  although  the  software 
tools  refereneed  in  the  report  are  under  USJFCOM  assessment  and  show 
potential,  the  software  tools  must  be  offieially  aeeredited  by  the  Offiee  of  the 
Seeretary  of  Defense  and  the  Joint  Staff  for  the  verifieation  of  joint 
interoperability  requirements  in  mission  area  arehiteetures. 

Co-Chair,  Interoperability  Senior  Review  Panel  Comments.  The  Co-Chair 
agreed  with  the  eoneept  of  the  reeommendation  and  provided  eomments  similar 
to  those  submitted  by  the  Inspeetor  General,  USJFCOM. 

Audit  Response.  The  Inspeetor  General,  USJFCOM  eomments  were 
responsive  to  the  intent  of  our  reeommendation.  However,  beeause  the  software 
tools  refereneed  in  the  report  are  under  USJFCOM  assessment  and  show 
potential,  but  have  not  yet  been  aeeredited,  we  redireeted  the  reeommendation  to 
the  Co-Chair,  Interoperability  Senior  Review  Panel  and  request  that  he  provide 
additional  eomments  on: 

•  when  the  aeereditation  proeess  for  the  software  tools  will  be 
eompleted,  and 
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•  when  the  software  tools  will  be  made  available  for  the  DoD 
Components  to  verify  joint  interoperability  requirements  of  planned 
weapon  systems  in  mission  area  arehiteetures. 

4.  We  recommend  that  the  Co-Chair,  Interoperability  Senior  Review 
Panel,  in  concert  with  the  applicable  membership  of  the  Interoperability 
Senior  Review  Panel: 

a.  Update  or  archive,  as  applicable,  ontdated  operational 
reqnirements  docnments  maintained  in  the  Joint  Command,  Control, 
Commnnications,  Compnters,  and  Intelligence  Program  Assessment  Tool 
database. 

Director,  Joint  Staff  Comments.  The  Director  concurred,  subject  to  the 
inclusion  of  his  comments,  and  suggested  that  the  recommendation  be  redirected 
to  the  ASD(C3I);  the  Director,  DISA;  and  the  Commander,  USJFCOM,  in 
addition  to  the  Director,  Command,  Control,  Communications,  and  Computers 
Systems  Directorate  (J-6),  Office  of  the  Chairman  of  the  Joint  Chiefs  of  Staff. 
He  stated  that  the  reason  for  redirecting  the  recommendation  was  because: 

•  JCPAT  access  and  archiving  are  functional  responsibilities  of  the 
Defense  Information  Systems  Agency; 

•  the  Office  of  the  ASD(C3I)  also  uses  the  JCPAT;  and 

•  as  the  warfighters  advocate  for  interoperability  and  integration, 
USJFCOM  should  participate  in  the  architecture  process  to  ensure 
that  warfighter  needs  and  interests  are  met. 

Co-Chair,  Interoperability  Senior  Review  Panel  Comments.  The  Co-Chair 
stated  that  the  Interoperability  Senior  Review  Panel  supports  the  position  of  the 
Joint  Staff  regarding  this  recommendation.  Specifically,  the  responsibility  for 
the  JCPAT  goes  beyond  the  Joint  Staff;  therefore,  the  recommendation  should 
be  addressed  to  all  departments,  agencies,  and  directorates  that  use  or  control 
access,  or  both,  to  the  JCPAT.  Further,  he  stated  that  the  Interoperability 
Senior  Review  Panel  will  review  the  policy  and  guidance  associated  with  the 
JCPAT  and  make  recommendations  to  appropriate  organizations. 

Audit  Response.  The  Director,  Joint  Staff  comments  were  responsive  to  the 
intent  of  our  recommendation.  In  response  to  those  comments,  we  redirected 
the  recommendation  as  suggested  by  the  Director,  Joint  Staff.  However, 
because  the  Offices  of  the  Chairman  of  the  Joint  Chiefs  of  Staff;  the  ASD(C3I); 
the  Commander,  USJFCOM;  and  the  Director,  DISA  are  represented  on  the 
Interoperability  Senior  Review  Panel,  we  redirected  the  recommendation  to  the 
Co-Chair,  Interoperability  Senior  Review  Panel  and  request  that  he  provide 
comments  on  how  those  represented  offices  will  update  or  archive,  as 
applicable,  outdated  ORDs  maintained  in  the  JCPAT  database. 


24 


b.  Limit  Joint  Command,  Control,  Commnnications,  Compnters, 
and  Intelligence  Program  Assessment  Tool  access  to  nsers  who  have  a  need 
to  know. 

Director,  Joint  Staff  Comments.  The  Director  concurred,  subject  to  the 
inclusion  of  his  comments,  and  suggested  that  the  recommended  corrective 
action  to  limit  JCPAT  access  to  users  who  have  a  need  to  know  be  coordinated 
among  the  ASD(C3I);  the  Director,  DISA;  and  Joint  Staff  directorates.  He 
stated  that  the  reason  for  coordinating  the  corrective  action  was  because  the 
JCPAT  is  a  collaborative  database  used  by  the  organizations  listed  in  his 
suggested  redirection  of  the  recommendation,  as  discussed  above  in 
Recommendation  4. a. 

Co-Chair,  Interoperability  Senior  Review  Panel  Comments.  The  Co-Chair’s 
comments  to  Recommendation  4. a.  also  applied  to  this  recommendation. 

Audit  Response.  The  Director,  Joint  Staff  comments  were  responsive  to  the 
intent  of  our  recommendation.  In  response  to  those  comments,  we  revised  and 
redirected  the  recommendation  as  he  suggested.  Therefore,  because  the  Offices 
of  the  Chairman  of  the  Joint  Chiefs  of  Staff;  the  ASD(C3I);  the  Commander, 
USJFCOM;  and  the  Director,  DISA  are  represented  on  the  Interoperability 
Senior  Review  Panel,  we  redirected  the  recommendation  to  the  Co-Chair, 
Interoperability  Senior  Review  Panel  and  request  that  he  provide  comments  on 
how  those  represented  offices  will  limit  JCPAT  access  to  users  having  a  need  to 
know. 

5.  We  recommend  that  the  Director,  Operational  Test  and  Evaluation 
define  and  implement  a  plan  to  test  critical  operational  issues,  including 
interoperability  and  information  assurance  requirements,  for  DoD  systems 
in  the  Global  Information  Grid. 

Director,  Operational  Test  and  Evaluation  Comments.  The  Director 
partially  concurred,  stating  that  testing  of  interoperability  and  information 
assurance  takes  on  more  critical  status  as  we  continue  to  transform  into  a 
network  centric  force.  Further,  he  stated  that  his  office  already  ensures  that 
interoperability  is  tested  to  support  the  assessment  of  key  performance 
parameters  provided  in  the  operational  requirements  documents  and  published 
policy  in  1999  for  testing  information  assurance. 

In  addition,  the  Director  stated  that  his  office  conducted  an  assessment  in  2000 
to  determine  requirements  for  interoperability  testing  of  systems  within  the 
scope  of  a  larger  networked  system  of  systems  linked  over  the  Global 
Information  Grid.  As  a  result  of  this  effort  and  similar  initiatives  by  the  Office 
of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
and  the  Joint  Staff,  DoD  funded  an  initial  capability  called  the  Joint  Distributed 
Engineering  Plant  in  the  FY  2002  Program  Objectives  Memorandum.  The 
Director  believes  that  the  objective  of  the  Joint  Distributed  Engineering  Plant 
capability  will  permit  his  office  to  do  the  testing  called  for  in  this 
recommendation.  However,  expanded  funding  for  the  Joint  Distributed 
Engineering  Plant  beyond  that  recommended  in  the  Program  Objectives 
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Memorandum  is  required  before  the  Plant  will  provide  the  full  seope  ealled  for 
in  this  reeommendation.  Further,  he  stated  that  interoperability  and  information 
assuranee  are  key  enablers  for  the  future. 

Co-Chair,  Interoperability  Senior  Review  Panel  Comments.  The  Co-Chair 
agreed,  providing  eomments  similar  to  those  by  the  Direetor,  Operational  Test 
and  Evaluation  eoneerning  testing  of  interoperability  and  information  assuranee 
and  testing  to  support  the  assessment  of  key  performanee  parameters  provided  in 
the  operational  requirements  doeuments.  Further,  he  stated  that  the 
Interoperability  Senior  Review  Panel  supports  the  development  of  neeessary  test 
eapabilities  to  represent  relevant  aspeets  of  the  Global  Information  Grid  that  will 
allow  programs  to  test  in  a  system  of  systems  environment.  The  Co-Chair  also 
stated  that  the  Offiees  of  the  Under  Seeretary  of  Defense  for  Aequisition, 
Teehnology,  and  Logisties;  the  ASD(C3I);  and  the  Direetor,  Operational  Test 
and  Evaluation  support  expanded  funding  in  the  upeoming  Program  Objeetives 
Memorandum  for  the  Joint  Distributed  Engineering  Plant,  whieh  will  allow 
eost-effeetive  testing  of  eritieal  operational  issues,  ineluding  interoperability  and 
information  assuranee  requirements,  for  DoD  systems  in  the  Global  Information 
Grid. 
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Appendix  A.  Scope  and  Methodology 


We  reviewed  doeumentation  dated  from  Mareh  1992  to  July  2002.  During  the 
audit  polieies  eontinued  to  be  updated.  To  aeeomplish  the  audit  objeetive,  we: 

•  interviewed  and  obtained  doeumentation  from  the  staffs  of  the  Under 
Seeretary  of  Defense  for  Aequisition,  Teehnology,  and  Logisties;  the 
Under  Seeretary  of  Defense  (Comptroller);  the  Assistant  Seeretary  of 
Defense  (Command,  Control,  Communieations,  and  Intelligenee);  the 
Direetor,  Operational  Test  and  Evaluation;  the  Offiee  of  the  Joint 
Chiefs  of  Staff;  the  U.S.  Joint  Forees  Command;  the  Defense 
Information  Systems  Ageney;  and  the  Joint  Interoperability  Test 
Command. 

•  reviewed  the  proeesses  that  the  Offiee  of  the  Seeretary  of  Defense, 
the  Offiee  of  the  Chairman  of  the  Joint  Chiefs  of  Staff,  and  the 
Defense  ageneies  used  for  eertifying  DoD  systems  for 
interoperability  and  information  assuranee  and  whether  those 
proeesses  were  effeetively  implementing  DoD  information 
interoperability  and  information  assuranee  polieies. 

Subsequent  reports  will  diseuss  the  adequaey  of  interoperability  key 
performanee  parameters  in  operational  requirements  doeuments  and  the 
eertifieation  proeess  for  DoD  system  interoperability  within  the  Army,  the 
Navy,  the  Air  Foree,  and  the  unified  eombatant  eommands. 

We  performed  this  audit  from  September  2001  through  July  2002  in  aeeordanee 
with  generally  aeeepted  government  auditing  standards.  We  did  not  review  the 
management  eontrol  program  beeause  the  audit  foeused  on  interoperability  and 
information  assuranee  requirements  and  review  proeesses;  therefore,  our  seope 
was  limited  to  those  speeifie  requirements  and  proeesses. 

General  Accounting  Office  High-Risk  Area.  The  General  Aeeounting  Offiee 
has  identified  several  high-risk  areas  in  the  DoD.  This  report  provides  eoverage 
of  the  DoD  weapon  systems  aequisition  high-risk  area. 

Use  of  Computer-Processed  Data.  We  did  not  rely  on  eomputer-proeessed 
data  to  perform  this  audit. 

Use  of  Technical  Assistance.  A  computer  specialist  from  the  Technical 
Assessment  Division,  Office  of  the  Assistant  Inspector  General  for  Auditing  of 
the  Department  of  Defense,  assisted  the  auditors  in  reviewing  information 
assurance  requirements. 
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Prior  Coverage 


During  the  last  5  years,  the  General  Aeeounting  Offiee  (GAO),  the  Inspeetor 
General  of  the  Department  of  Defense  (IG  DoD),  and  the  Defense  Seienee 
Board  have  issued  five  reports  addressing  interoperability  and  information 
assuranee  requirements  for  Defense  systems.  Unrestrieted  General  Aeeounting 
Offiee  and  Inspeetor  General  of  the  Department  of  Defense  reports  ean  be 
aeeessed  at  http ;//www. gao.gov  and  http ;//www. dodig.osd.mil/audit/reports, 
respeetively. 

General  Accounting  Office 

GAO  Report  No.  NSIAD-98-73,  “Joint  Military  Operations:  Weakness 
in  DoD’s  Proeess  for  Certifying  C4I  Systems’  Interoperability,”  Mareh 
1998 

Inspector  General  of  the  Department  of  Defense 

IG  DoD  Report  No.  D-2001-176,  “Survey  of  Aequisition  Manager 
Experienee  using  the  DoD  Joint  Teehnieal  Arehiteeture  in  the 
Aequisition  Proeess,”  August  22,  2001 

IG  DoD  Report  No.  D-2001-121,  “Use  of  the  DoD  Joint  Teehnieal 
Arehiteeture  in  the  Aequisition  Proeess,”  May  14,  2001 

IG  DoD  Report  No.  98-023,  “Implementation  of  the  DoD  Joint 
Teehnieal  Arehiteeture,”  November  18,  1997 

Defense  Science  Board 

Defense  Seienee  Board  Task  Foree,  “Proteeting  the  Homeland,  Report 
of  the  Defense  Seienee  Board  Task  Foree  on  Defensive  Information 
Operations,  2000  Summer  Study,  Volume  II,”  Mareh  2001 


Other  Matters  of  Interest 


Key  Performance  Parameter.  Although  information  assurance  is  not  required 
to  be  a  KPP  in  operational  requirements  documents,  without  them,  milestone 
decision  authorities  increase  the  risk  that  weapon  systems  program  managers 
will  not  achieve  secure  information  exchanges  before  planned  production 
decisions.  We  suggested  to  the  Joint  Staff  (J-8)  that  information  assurance  be 
established  as  mandatory  KPP.  The  J-8  responded  that  interoperability  was  the 
only  mandatory  KPP  and  that  KPPs  were  driven  by  the  capabilities  deemed  most 
essential  by  the  warfighter  and  validated  by  the  requirements  authority  in 
accordance  with  guidance  in  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruction  3170.01B,  “Requirements  Generation  System,”  April  15,  2001. 
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Furthermore,  the  J-8  stated  that  the  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruetion  3 170.0 IB  does  require  information  assuranee  for  most  information 
teehnology  systems  and,  in  some  eases,  information  assuranee  has  been  made  a 
KPP.  For  example,  the  Global  Information  Grid  Capstone  Requirement 
Doeument  requires  information  assuranee  as  a  KPP.  Therefore,  ORDs  for 
programs  operating  within  the  GIG  must  have  information  assuranee 
requirements,  ineluding  designating  information  assuranee  as  a  KPP,  as 
appropriate. 

Information  Assnrance  Strategies.  DoD  poliey  requires  ehief  information 
offieers  to  review  and  eonfirm  that  the  information  assuranee  strategy  is 
eonsistent  with  DoD  polieies,  standards,  and  arehiteetures.  The  ASD(C3I)  is 
required  to  review  the  information  assuranee  strategy  for  all  major  DoD  systems 
before  the  milestone  deeision  authority  approves  the  DoD  system  for  entry  into 
the  next  aequisition  phase.  The  ASD(C3I)  uses  a  eheeklist  based  on  the  Defense 
Information  Teehnology  Seeurity  Certifieation  Aeereditation  Proeess  to  review 
information  assuranee  strategies. 

Over  the  past  2  years,  the  ASD(C3I)  has  reviewed  and  had  input  on 
approximately  40  programs.  The  ASD(C3I)  eould  not  provide  the  number  of 
information  assuranee  strategies  that  had  been  disapproved,  but  eommented  that 
the  goal  was  to  work  with  program  managers  toward  sueeessful  information 
assuranee  strategies.  For  non-aequisition  DoD  systems  sueh  as  those  developed 
from  advaneed  eoneept  teehnology  demonstrations,  the  designated  milestone 
deeision  authority  is  required  to  eertify  that  the  system  eomplies  with  the 
Defense  Information  Teehnology  Seeurity  Certifieation  Aeereditation  Proeess 
proeedures  before  approving  the  system  for  fielding.  Aeeordingly,  ehief 
information  offieers,  program  managers,  and  testers  must  be  knowledgeable  of 
those  proeedures.  However,  a  requirement  did  not  exist  for  the  ASD(C3I)  to 
provide  oversight  for  those  non-aequisition  DoD  systems  to  verify  that  the 
systems  had  an  information  assuranee  strategy  and  were  aeeredited. 

Beginning  in  July  2002,  the  National  Seeurity  T el ecommun i cati on  and 
Information  Systems  Seeurity  Poliey  requires  that  all  U.  S.  Government 
departments  limit  the  aequisition  of  commercial  off-the-shelf  information 
assurance  and  information  assurance-enabled  products  to  those  that  have  been 
evaluated  and  validated  in  accordance  with  the; 

•  Common  Criteria, 

•  National  Information  Assurance  Partnership  Evaluation  and 
Validation  Program,  and 

•  Federal  Information  Processing  Standards  Publications  Validation 
Program. 

The  ASD(C3I)  stated  that  chief  information  officers,  program  managers,  and 
testers  in  Defense  agencies  and  Military  Departments  will  have  to  execute  the 
policy. 
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Appendix  B.  Definitions  of  Technical  Terms 


Acquisition  Category.  An  acquisition  category  determines  an  aequisition 
program’s  level  of  review,  deeision  authority,  and  applieable  proeedures.  The 
aequisition  eategories  eonsist  of  I,  major  Defense  aequisition  programs;  lA, 
major  automated  information  systems;  II,  major  systems;  III,  programs  not 
meeting  the  eriteria  for  aequisition  eategories  I,  lA,  or  II;  and  IV,  programs 
designated  as  sueh  by  the  Army,  Navy,  and  Marine  Corps. 

Advanced  Concept  Technology  Demonstration.  An  advanced  concept 
technology  demonstration  is  used  to  determine  the  military  utility  of  proven 
technology  and  to  develop  the  concept  of  operations  that  will  optimize 
effectiveness.  Advanced  concept  technology  demonstrations  are  not  themselves 
acquisition  programs,  but  are  designed  to  provide  a  residual,  usable  capability 
upon  completion,  and  possibly  transition  into  acquisition  programs.  Funding  is 
programmed  to  support  the  demonstration  for  up  to  2  years  in  the  field. 

Architecture.  An  architecture  is  the  structure  of  components,  their 
interrelationships,  and  the  principal  guidelines  governing  their  design  and 
evolution  over  time. 

Capstone  Requirements  Document.  A  capstone  requirements  document  is  a 
document  that  contains  capabilities-based  requirements  for  developing  individual 
ORDs  by  providing  a  common  framework  and  operational  concept  to  guide  their 
development.  It  is  an  oversight  tool  containing  overarching  requirements  for  a 
system-of-systems  or  family-of-systems. 

Combat  Developer.  A  combat  developer  is  the  command  or  agency  that 
formulates  doctrine,  concepts,  organization  makeup,  materiel  requirements,  and 
objectives.  Generically,  it  represents  the  user  community  role  in  the  materiel 
acquisition  process. 

Command,  Control,  Communications,  Computers,  and  Intelligence  Support 

Plan.  A  C4I  support  plan  describes  system  dependencies  and  interfaces  in 
sufficient  detail  to  enable  program  managers  and  operational  testers  to  test 
interoperability  key  performance  parameters  derived  from  information  exchange 
requirements. 

Command,  Control,  Communications,  Computers,  and  Intelligence 
Surveillance  and  Reconnaissance  Architecture  Framework.  The  C4I 

surveillance  and  reconnaissance  architecture  framework  provides  rules, 
guidance,  and  product  descriptions  for  developing  and  presenting  different 
architectural  views  of  a  given  system  to  ensure  a  common  denominator  for 
understanding,  comparing,  and  integrating  architectures  across  DoD. 

Critical  Operational  Issue.  A  critical  operational  issue  is  a  key  operational 
effectiveness  issue  or  operational  suitability  issue  that  must  be  examined  in  the 
operational  test  and  evaluation  to  determine  the  system’s  capability  to  perform 
its  mission. 
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Defense  Acquisition  Executive  Summary.  The  Defense  Aequisition  Exeeutive 
Summary  is  a  multi-part  doeument,  whieh  reports  program  information  and 
assessments;  program  manager,  program  exeeutive  offiee,  and  Component 
aequisition  exeeutive  eomments;  and  eost  and  funding  data  primarily  for 
Aequisition  Category  I  programs. 

Defense-in-Depth.  Defense-in-depth  integrates  the  eapabilities  of  people, 
operations,  and  teehnology  to  establish  multi-layer,  multi-dimension  proteetion 
of  networked  systems.  The  eoneept  is  to  deploy  defenses  at  multiple  loeations 
in  sueeessive  layers  in  the  proteeted  information  environment.  Defense-in¬ 
depth  foeuses  on  the  loeal  eomputing  environments  (or  enelaves),  enelave 
boundaries,  networks  that  link  enelaves,  and  supporting  infrastruetures. 

Developmental  Test  and  Evaluation.  Developmental  test  and  evaluation  is  any 
engineering  type  of  test  used  to  verify  the  status  of  teehnieal  progress,  verify 
that  design  risks  are  minimized,  substantiate  aehievement  of  eontraet  teehnieal 
performanee,  and  eertify  readiness  for  initial  operational  testing.  Generally, 
those  tests  are  instrumented  and  measured  by  engineers,  teehnieians,  or  soldier 
operator-maintainer  test  personnel  in  a  eontrolled  environment  to  faeilitate 
failure  analysis. 

Global  Information  Grid.  The  Global  Information  Grid  provides  the 
foundation  for  network-eentrie  warfare,  information  superiority,  deeision 
superiority,  and  ultimately,  full  speetrum  dominanee.  The  GIG  ineludes  any 
system,  equipment  software,  or  serviee  that  transmits  information  to,  reeeives 
information  from,  routes  information  among  or  interehanges  information  among 
other  equipment,  software,  and  serviees.  Non-GIG  Information  Teehnology  is 
stand-alone,  self-eontained,  or  embedded  information  teehnology  that  is  not  or 
will  be  eonneeted  to  the  enterprise  network. 

Information  Assurance.  Information  assurance  is  information  operations  that 
protect  and  defend  the  information  and  information  systems  by  ensuring  their 
availability,  integrity,  authenticity,  confidentiality,  and  nonrepudiation. 
Information  assurance  provides  for  the  restoration  of  information  systems  by 
incorporating  protection,  detection,  and  reaction  capabilities. 

Information  Exchange  Requirements.  Information  exchange  requirements 
characterize  the  information  exchanges  to  be  performed  by  a  proposed  system 
and  identify  who  exchanges  what  information  with  whom,  why  the  information 
is  necessary,  and  how  the  users  will  employ  that  information. 

Information  Technology.  Information  technology  includes  any  equipment  or 
interconnected  system  or  subsystem  of  equipment  that  is  used  in  automatic 
acquisition,  storage,  manipulation,  management,  movement,  control,  display, 
switching,  interchange,  transmission,  or  reception  of  data  or  information. 
Information  technology  includes  computers,  software,  firmware,  ancillary 
equipment  and  similar  procedures,  services,  and  related  resources. 

Information  Warfare.  Information  warfare  is  used  to  achieve  information 
superiority  by  affecting  adversary  information,  information-based  processes. 
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information  systems,  and  computer-based  networks  while  defending  one’s  own 
information,  information-based  processes,  information  systems,  and 
computer-based  networks. 

Interoperability.  Interoperability  is  the  ability  of  systems,  units,  or  forces  to 
provide  services  to  or  accept  services  from  other  systems,  units,  or  forces  and  to 
use  the  services  so  exchanged  to  operate  effectively  together. 

Interoperability  Certification  Process.  The  interoperability  certification 
process  consists  of  two  interoperability  certifications;  the  first  for  requirements 
and  supportability  documents  and  the  second  for  interoperability  certification 
testing.  The  Chairman  of  the  Joint  Chiefs  of  Staff  (J-6)  and  the  Joint 
Interoperability  Test  Command  are  responsible  for  requirements  and 
supportability  document  certification  and  for  interoperability  certification 
testing,  respectively. 

Interoperability  Senior  Review  Panel.  The  Interoperability  Senior  Review 
Panel  recommends  programs  and  systems  deemed  to  have  significant 
interoperability  deficiencies  for  the  Interoperability  Watch  List.  The  Panel  is 
composed  of  the  staffs  from  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics;  the  ASD(C3I);  the  Director,  Operational  Test  and 
Evaluation;  the  USJFCOM,  and  the  Joint  Staff. 

Joint  Distributed  Engineering  Plant.  The  Joint  Distributed  Engineering  Plant 
is  a  DoD-wide  effort  to  link  existing  Defense  agencies  and  Military  Departments 
and  joint  combat  system  engineering  and  test  sites  (including  design  activities, 
software  support  activities,  test  and  evaluation  facilities,  training  commands, 
and  operational  units).  The  Joint  Distributed  Engineering  Plant  is  designed  to 
improve  the  interoperability  of  weapon  systems  and  platforms  through  rigorous 
testing  and  evaluation  in  a  replicated  battlefield  environment. 

Joint  Mission  Area.  A  joint  mission  area  is  a  functional  group  of  joint  tasks 
and  activities  that  share  a  common  purpose  and  facilitate  joint  force  operations. 

Joint  Operational  Architecture.  A  joint  operational  architecture  describes 
tasks  and  activities,  operational  elements,  and  information  flows  required  to 
accomplish  or  support  military  operations;  defines  types  of  information 
exchanged,  frequency  of  exchange,  which  tasks  and  activities  are  supported  by 
information  exchanges,  and  nature  of  information  exchanges  in  detail  sufficient 
to  ascertain  specific  interoperability  requirements. 

Joint  Requirements  Oversight  Council.  The  Joint  Requirements  Oversight 
Council  assists  the  Chairman  of  the  Joint  Chiefs  of  Staff  in  identifying  and 
assessing  the  priority  of  joint  military  requirements  (including  existing  systems 
and  equipment)  to  meet  the  national  military  strategy.  The  council,  chaired  by 
the  Vice  Chairman  of  the  Joint  Chiefs  of  Staff  and  consisting  of  all  the  Vice 
Chiefs  of  the  Military  Departments  including  the  Assistant  Commandant  of  the 
Marine  Corps,  directly  supports  the  Defense  Acquisition  Board  through  review, 
validation,  and  approval  of  key  cost,  schedule,  and  performance  parameters  at 
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the  start  of  the  aequisition  proeess,  before  eaeh  milestone  review,  and  as 
requested  by  the  Under  Seeretary  of  Defense  for  Aequisition,  Teehnology,  and 
Logisties. 

Joint  Systems  Architecture.  The  joint  systems  arehiteeture  identifies  and 
deseribes  those  DoD  systems  and  their  interconnections  needed  to  accomplish 
the  tasks  and  activities  described  in  the  Joint  Operational  Architecture. 

Joint  Technical  Architecture.  The  joint  technical  architecture  is  a  common  set 
of  mandatory  information  technology  standards,  which  are  primarily  interface 
standards  and  guidelines  to  be  used  by  all  emerging  systems  and  systems 
upgrades,  including  advanced  concept  technology  demonstrations.  The  joint 
technical  architecture  can  be  used  to  establish  a  system’s  technical  architecture, 
and  is  applicable  to  all  C4I  and  automated  information  systems  and  the 
interfaces  of  other  key  assets,  such  as  weapon  systems  and  sensors,  with  C4I 
systems. 

Key  Performance  Parameters.  Key  performance  parameters  are  capabilities 
or  characteristics  so  significant  that  failure  to  meet  the  threshold  or  minimum 
acceptable  value  can  be  cause  for  the  concept  or  system  selected  to  be 
reevaluated  or  for  the  program  to  be  reassessed  or  terminated. 

Measure  of  Effectiveness.  A  measure  of  effectiveness  is  the  quantitative 
expression  defined  to  measure  operational  capabilities  in  terms  of  engagement  of 
battle  outcomes. 

Measures  of  Performance.  Measures  of  performance  measure  a  system’s 
technical  performance  expressed  as  speed,  payload,  range,  time  on  station, 
frequency,  or  other  distinctly  quantifiable  performance  features.  Several 
measures  of  performance  may  be  related  to  the  achievement  of  a  particular 
measure  of  effectiveness. 

National  Security  System.  A  national  security  system  is  any 
telecommunication  or  information  system  operated  by  the  United  States 
Government,  whose  function,  operation,  or  use  involves  intelligence  activities, 
cryptologic  activities  related  to  national  security,  command  and  control  of 
military  forces,  equipment  that  is  an  integral  part  of  a  weapon  system,  or  is 
critical  to  the  direct  fulfillment  of  military  or  intelligence  missions. 

Network-Centric  Warfare.  Network-centric  warfare^"^  allows  a  warfighting 
force  to  achieve  improved  information  positions  in  the  form  of  common 
operational  pictures  that  provide  the  basis  for  shared  situational  awareness  and 
knowledge,  and  a  resulting  increase  in  combat  power. 

Operational  Architecture  View.  The  operational  architecture  view  is  a 
description  of  the  tasks  and  activities,  operational  elements,  and  information 


in-depth  discussion  of  network-centric  warfare  is  provided  in  the  book,  Network  Centric  Warfare: 
Developing  and  Leveraging  Information  Superiority,  2”“'^ Edition  (Revised),  by  David  S.  Alberts,  John  J. 
Garstka,  and  Frederick  P.  Stein,  C^I  Surveillance  and  Reconnaissance  Cooperative  Research  Program, 
August  1999. 
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flows  required  to  aeeomplish  or  support  a  military  operation.  This  arehiteeture 
view  defines  the  types  of  information  exehanged,  the  frequeney  of  exehange, 
whieh  tasks  and  aetivities  are  supported  by  the  information  exehanges,  and  the 
nature  of  information  exehanges  in  detail  suffieient  to  aseertain  speeifie 
interoperability  requirements. 

Operational  Effectiveness.  Operational  effeetiveness  is  the  overall  degree  of 
mission  aeeomplishment  of  a  system  when  representative  personnel  use  the 
system  in  the  environment  planned  or  expeeted  for  operational  employment  of 
the  system,  eonsidering  organization,  doetrine,  taeties,  survivability, 
vulnerability,  and  threat. 

Operational  Reqnirements  Docnment.  The  operational  requirements 
doeument  states  the  user’s  objeetives  and  minimum  aeeeptable  requirements  for 
the  operational  performanee  of  a  proposed  eoneept  or  system. 

Operational  Snitability.  Operational  suitability  is  the  degree  to  whieh  a  system 
ean  be  plaeed  satisfaetorily  in  field  use  with  eonsideration  being  given  to 
availability,  eompatibility,  transportability,  interoperability,  reliability,  wartime 
usage  rates,  maintainability,  safety,  human  faetors,  manpower  supportability, 
logistie  supportability,  natural  environmental  effeets,  doeumentation,  and 
training  requirements. 

Operational  Test  and  Evalnation.  Operational  test  and  evaluation  is  field 
testing,  under  realistie  eonditions,  of  any  item  or  eomponent  of  weapons, 
equipment,  or  munitions  to  determine  their  effeetiveness  and  suitability  for  use 
in  eombat  by  typieal  military  users  and  the  evaluation  of  the  results  of  sueh 
tests. 

Planning,  Programming,  and  Bndgeting  System.  The  Planning, 
Programming,  and  Budgeting  System  is  the  primary  resouree  alloeation  proeess 
of  DoD.  It  is  a  formal,  systematie  strueture  for  making  deeisions  on  poliey, 
strategy,  and  the  development  of  forees  and  eapabilities  to  aeeomplish 
antieipated  missions. 

Program.  A  program  is  an  aequisition  funded  by  researeh,  development,  test 
and  evaluation  or  proeurement  appropriations,  or  both,  with  the  express 
objeetive  of  providing  a  new  or  improved  eapability  in  response  to  a  stated 
mission  need  or  defieieney. 

Program  Objectives  Memorandum.  The  Program  Objeetives  Memorandum  is 
an  annual  memorandum,  whieh  the  DoD  Component  Heads  submit  to  the 
Seeretary  of  Defense,  that  reeommends  the  total  resouree  requirements  and 
programs  within  the  parameters  of  the  Seeretary  of  Defense’s  fiseal  guidanee. 

Risk.  Risk  is  a  eombination  of  the  likelihood  that  a  threat  will  oeeur,  the 
likelihood  that  a  threat  oeeurrenee  will  result  in  an  adverse  impaet,  and  the 
severity  of  the  resulting  adverse  impaet. 
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System.  A  system  is  the  organization  of  hardware,  software,  materiel, 
faeilities,  personnel,  data,  and  serviees  needed  to  perform  a  designated  funetion 
with  speeified  results,  sueh  as  the  gathering  of  speeified  data,  its  proeessing,  and 
delivery  to  users. 

System-of-Systems.  System-of-systems,  also  known  as  a  family-of-systems,  is 
several  independent  programs  whieh,  when  integrated,  form  a  system  to  meet 
the  needs  of  a  broad  mission  area  sueh  as  missile  defense.  The  performanee  of 
the  individual  eomponent  programs  making  up  the  system-of-systems  is 
speeified  in  the  respeetive  program  ORDs;  the  overarehing  requirements  for  the 
system-of-systems  is  eontained  in  a  CRD. 

Systems  Architecture  View.  The  systems  arehiteeture  view  is  a  deseription, 
ineluding  graphies,  of  systems  and  intereonneetions  providing  for,  or 
supporting,  warfighting  funetions.  This  arehiteeture  view  assoeiates  physieal 
resourees  and  their  performanee  attributes  to  the  operational  view  and  its 
requirements  in  aeeordanee  with  standards  defined  in  the  teehnieal  arehiteeture. 

Technical  Architecture  View.  The  teehnieal  arehiteeture  view  is  the  minimum 
set  of  rules  governing  the  arrangement,  interaetion,  and  inter dependenee  of 
system  parts  or  elements,  whose  purpose  is  to  ensure  that  a  eonformant  system 
satisfies  a  speeified  set  of  requirements.  This  arehiteeture  view  ineludes  a 
eolleetion  of  the  teehnieal  standards,  eonventions,  rules,  and  eriteria  organized 
into  profile(s)  that  govern  system  serviees,  interfaees,  and  relationships  for 
partieular  systems  arehiteeture  views  and  that  relate  to  partieular  operational 
views. 

Test  and  Evaluation  Master  Plan.  The  test  and  evaluation  master  plan 
(TEMP)  doeuments  the  overall  strueture  and  objeetives  of  the  test  and 
evaluation  program.  It  provides  a  framework  within  whieh  to  generate  detailed 
test  and  evaluation  plans  and  it  doeuments  sehedule  and  resouree  implieations 
assoeiated  with  the  test  and  evaluation  program.  The  TEMP  identifies  the 
neeessary  developmental  test  and  evaluation,  operational  test  and  evaluation,  and 
live-fire  test  and  evaluation  aetivities.  Eurther,  the  TEMP  relates  program 
sehedule,  test  management  strategy  and  strueture,  and  required  resourees  to 
eritieal  operational  issues,  eritieal  teehnieal  parameters,  objeetives  and 
thresholds  doeumented  in  the  operational  requirements  doeument,  evaluation 
eriteria,  and  milestone  deeision  points. 

Test  Integration  Working  Group.  The  Test  Integration  Working  Group 
faeilitates  the  integration  of  test  requirements  through  elose  eoordination 
between  the  materiel  developer,  eombat  developer,  logistieian,  and 
developmental  and  operational  testers  to  minimize  development  time  and  eost, 
and  preelude  duplieation  between  developmental  and  operational  testing. 
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Appendix  C.  Interoperability  Action  Plan  of  the 

Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and 
Logistics  for  Command  and 
Control  Systems^^ 


Policy 

modifications 
and  transition 
plans  in  place 


Overarching  battle 
management/ 
command  and 
control  begin  to 
take  effect 


“Readily  Resolvable” 
interoperability 
problems  solved  for 
legacy  command  and 
control  systems 


control 

mteroperability 
problems  resolved; 
Interoperability 
institutionalized  in 
processes  and 
Architectures 


'^The  numbers  in  the  diagram  relating  to  the  elements  required  to  achieve  interoperability  are  described 
on  the  next  page. 
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According  to  the  Director  of  Interoperability,  Offiee  of  the  Under  Seeretary  of 
Defense  for  Aequisition,  Teehnology,  and  Logisties,  the  following  four  major 
elements  are  needed  to  aehieve  interoperability  for  eommand  and  eontrol 
systems. 

1.  Doetrine,  organization,  training,  taeties,  teehniques  and  proeedures, 
leadership,  personnel,  faeilities,  whieh  form  the  foundation  for  any 
military  operation. 

2.  Polieies,  direetives,  proeesses,  standards,  and  arehiteetures  without 
whieh  real  progress  is  impossible.  Polieies  are  neeessary;  however 
they  are  not  suffieient. 

3.  To  transition  from  legaey  DoD  systems  to  the  future  vision, 
overarehing  battle  management  eommand  and  eontrol  programs  are 
neeessary  to  provide  for  legaey  DoD  systems’  integration  and 
transition  to  the  desired  interoperability  state.  Those  overarehing 
programs  are: 

•  The  Family  of  Interoperable  Operational  Pietures  (FIOP),  whieh 
will  provide  for  systems  engineering  solutions  for  aehieving 
interoperability  of  more  than  30  eritieal  battle  management 
eommand  and  eontrol  systems  (fielded  and  in  development), 
within  appropriate  “eommunities  of  shared  interest,”  from  the 
eombatant  eommander  to  the  soldier,  sailor,  marine,  airman,  and 
eoalition  partners,  ineluding  the  seams  between  the  Single 
Integrated  Air  Pieture,  a  Single  Integrated  Ground  Pieture,  a 
Single  Integrated  Maritime  Pieture,  and  the  Common  Operational 
Pieture/Common  Taetieal  Pieture. 

•  The  Joint  Interoperability  and  Integration  (III)  reviews  legaey  and 
future  systems’  interoperability  requirements  in  the  total  eontext 
of  doetrine,  operations,  training,  materiel,  logisties,  personnel, 
and  faeilities,  and  is  tied  to  an  “Interoperability  Transition  Fund” 
intended  to  foeus  on  priority  interoperability  fixes. 

•  The  Joint  Distributed  Engineering  Plant  (JDEP)  is  designed  to 
improve  interoperability  through  distributed  testing  and 
evaluation  involving  the  Serviees,  DoD  Components,  and 
eventually  industrial  and  allied  resourees. 

4.  New  systems  sueh  as  the  Joint  Taetieal  Radio  System  will  eontribute 
to  the  interoperability  solution.  Many  marginally  interoperable 
legaey  systems  will  be  phased  out.  The  Quadrennial  Defense  Review 
and  the  normal  Planning,  Programming,  and  Budgeting  System 
proeess  should  be  used  to  determine  priorities  and  funding  for  those 
systems. 

The  Under  Seeretary  of  Defense  for  Aequisition,  Teehnology,  and  Logisties, 
estimates  that  if  the  overarehing  programs  did  not  transition  from  the  legaey  to 
the  future  vision,  the  time  to  aehieve  the  desired  interoperability  state  would 
most  likely  double. 
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Appendix  D.  Global  Information  Grid 


Global  Information  Grid.  The  GIG  provides  the  foundation  for  network- 
eentrie  warfare,  information  superiority,  deeision  superiority,  and  ultimately  full 
speetrum  dominanee  as  depleted  below. 


Global  Information  Grid  as  the  Foundation  for  Achieving  Full  Spectrum 
Dominance 

Source:  Capstone  Requirements  Document  for  the  GIG 


The  eoneept  of  the  GIG  evolved  from  eoneerns  about  the  interoperability  and 
end-to-end  integration  of  automated  information  systems.  Issues  sueh  as 
streamlined  management  and  improved  information  infrastrueture  investment 
also  eontributed  to  the  heightened  interest  in  a  GIG.  However,  the  real  demand 
for  a  GIG  originates  from  the  requirement  for  information  and  deeision 
superiority  to  aehieve  full  speetrum  dominanee,  as  expressed  in  Joint 
Vision  2020.  The  ability  to  aehieve  shared  situational  awareness  and  knowledge 
among  all  elements  of  a  joint  foree,  ineluding  allied  and  eoalition  partners,  is 
inereasingly  viewed  as  a  eornerstone  to  transform  future  warfighting 
eapabilities. 

Network-Centric  Warfare.  The  GIG  eapstone  requirement  doeument  states 
that  network-eentrie  warfare  allows  a  warfighting  foree  to  aehieve  improved 
information  positions  in  the  form  of  eommon  operational  pietures  that  provide 
the  basis  for  shared  situational  awareness  and  Imowledge,  and  a  resulting 
inerease  in  eombat  power. 

Information  Superiority.  Information  superiority  is  the  eapability  to  eolleet, 
proeess,  and  disseminate  an  uninterrupted  flow  of  information  while  exploiting 
or  denying  an  adversary’s  ability  to  do  the  same.  Information  superiority  is 
aehieved  in  a  noneombat  situation  or  one  in  whieh  there  are  no  elearly  defined 
adversaries  when  friendly  forees  have  the  information  neeessary  to  aehieve 
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operational  objectives.  Information  superiority  provides  the  joint  force  with  a 
competitive  advantage  only  when  it  is  effectively  translated  into  superior 
knowledge  and  decisions.  The  joint  force  must  be  able  to  take  advantage  of 
superior  information  converted  to  superior  knowledge  to  achieve  “decision 
superiority.  ” 

Decision  Superiority.  Decision  superiority  is  to  arrive  at  better  decisions  and 
implemented  faster  than  an  opponent  can  react,  or  in  a  noncombat  situation,  at  a 
tempo  that  allows  the  force  to  shape  the  situation  or  react  to  changes  and 
accomplish  its  mission.  Decision  superiority  does  not  automatically  result  from 
information  superiority.  Organizational  and  doctrinal  adaptation,  relevant 
training  and  experience,  and  the  proper  command  and  control  mechanisms  and 
tools  are  equally  necessary. 

Full  Spectrum  Domiuauce.  The  transformation  of  the  joint  force  to  reach  full 
spectrum  dominance  rests  upon  information  superiority  as  a  key  enabler  and  our 
capacity  for  innovation.  The  label  full  spectrum  dominance  implies  that  U.S. 
Forces  are  able  to  conduct  prompt,  sustained,  and  synchronized  operations  with 
combinations  of  forces  tailored  to  specific  situations  and  with  access  to  and 
freedom  to  operate  in  all  domains:  space,  sea,  land,  air,  and  information. 
Additionally,  given  the  global  nature  of  our  interests  and  obligations,  the  United 
States  must  maintain  its  overseas  presence  forces  and  the  ability  to  rapidly 
project  power  worldwide  in  order  to  achieve  full  spectrum  dominance. 
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Appendix  E.  Multiple  Factors  Affecting  the 

Implementation  of  Interoperability 
Requirements  in  the  Weapon 
System  Development  Process 


The  following  diagram  shows  the  eomplex  relationships  governing  the 
development  of  an  interoperable  system.  The  diagram  shows  four  relationships 
for  aehieving  interoperability. 

•  People  and  Organizations.  Organizations  with  responsibility  for 
verifying  and  validating  interoperability  requirements  before 
produetion  deeisions  are  shown  in  the  diagram.  Milestone  deeision 
authorities  ean  not  approve  DoD  systems  for  produetion  before 
interoperability  requirements  are  determined  and  tested. 

•  Policies.  Program  managers  of  DoD  systems  need  to  eomply  with 
numerous  polieies,  doetrines,  and  arehiteetures  that  define  how  the 
systems  must  operate  in  different  environments  and  eireumstanees 
around  the  world.  Therefore,  eombat  developers  must  elearly  depiet 
the  doetrine,  training,  organizations,  soldiers,  faeilities,  and 
relationship  of  the  system  to  joint  and  allied  systems  before 
developing  and  aequiring  a  system. 

•  Integrated  Testing  Environments.  Military  Departments  need  test 
faeilities  and  equipment  to  test  the  aehievement  of  interoperability 
requirements  for  a  DoD  system  in  development.  To  realistieally  test 
how  a  system  will  interoperate  in  a  joint  and  allied  environment  as 
envisioned  in  the  GIG  and  Joint  Vision  2020,  additional  equipment 
and  resourees  may  be  required  or  shared  with  integrated  testing 
environments. 

•  Cost.  The  Planning,  Programming,  and  Budgeting  System  is  a 
struetured  proeess  that  results  in  funding  for  DoD  systems.  Materiel 
developers  of  DoD  systems  attempt  to  maintain  the  planned 
aequisition  strategy.  If  the  aequisition  strategy  is  not  maintained, 
adjustments  to  the  funding  proeess  should  be  made  that  are  not 
always  possible  beeause  of  other  DoD  funding  priorities. 

All  of  the  relationships  must  be  met  for  a  DoD  system  to  meet  the  user’s 
requirement  for  interoperability. 
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Multiple  Factors  Affecting  the  Implementation 
of  Interoperability  Requirements  in  the 
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Acronyms 


ACTD 

CINC 

CJCSI 

CIO 

DODI 

DOT&E 

JCS 

J-3 

J-6 


J-8 


JITC 

PPBS 

USD(AT&L) 


USD  (Comptroller) 
USJFCOM 


Advanced  Concept  Technology  Demonstration 
Commander-in-Chief 

Chairman  of  the  Joint  Chiefs  of  Staff  Instruction 
Chief  Information  Officer 
Department  of  Defense  Instruction 
Director,  Operational  Test  and  Evaluation 
Joint  Chiefs  of  Staff 

Director  for  Operations,  Joint  Chiefs  of  Staff 
Director  for  Command,  Control,  Communications  and  Computers 
Systems,  Joint  Chiefs  of  Staff 

Director  for  Force  Structure,  Resources,  and  Assessments,  Joint 
Chiefs  of  Staff 

Joint  Interoperability  Test  Command 
Plaiming,  Programming,  and  Budgeting  System 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics 

Under  Secretary  of  Defense  (Comptroller)/Chief  Financial  Officer 
U.S.  Joint  Forces  Command 
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Appendix  F.  Interoperability  and  Information 

Assurance  Policies 


The  following  discusses  relevant  DoD  and  Joint  Staff  policies  on  interoperability 
and  information  assurance. 

Interoperability  Policy.  DoD  Directive  4630.5,  “Interoperability  and 
Supportability  of  Information  Technology  (IT)  and  National  Security  Systems 
(NSS)”  January  11,  2002;  DoD  Instruction  4630.8,  “Procedures  for 
Interoperability  and  Supportability  of  Information  Technology  (IT)  and  National 
Security  Systems  (NSS),”  May  2,  2002;  DoD  Instruction  5000.2,  “Operation  of 
the  Defense  Acquisition  System,”  April  5,  2002;  DoD  Regulation  5000. 2-R, 
“Mandatory  Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  and 
Major  Automated  Information  System  (MAIS)  Acquisition  Programs,”  April  5, 
2002;  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3170. OIB, 

“Requirements  Generation  System,”  April  15,  2001;  and  Chairman  of  the  Joint 
Chiefs  of  Staff  Instruction  62 12.0 IB,  “Interoperability  and  Supportability  of 
National  Security  Systems,  and  Information  Technology  Systems,”  May  8, 

2000,  provide  policy  on  interoperability. 

DoD  Directive  4630.5.  DoD  Directive  4630.5  requires  that  information 
technology  and  national  security  systems’  interoperability  and  supportability 
needs  be  identified  through  the  requirements  definition  and  validation  process, 
during  the  acquisition  process,  and  updated  as  necessary  throughout  the  system’s 
life.  Those  requirements  will  be  specified  to  a  level  of  detail  that  allows 
verification  of  interoperability  throughout  a  system’s  life.  Also,  those 
requirements  will  be  characterized  through  operational  mission  area  integrated 
architectures,  operational  concepts,  and  capstone  requirements  documents 
(CRDs)  derived  from  joint  mission  areas  and  business  and  administrative 
mission  areas.  The  joint  operational  architecture,  the  joint  systems  architecture, 
and  the  joint  technical  architecture  serve  as  the  foundation  for  development  of 
operational  mission  area  integrated  architectures.  Operational  mission  area 
integrated  architectures  connect  information  technology  and  the  national  security 
systems’  interoperability  and  supportability  requirements  in  a  family-of-systems 
and  system-of-systems  context.  Information  technology  and  national  security 
systems’  information  exchange  requirements  and  associated  interoperability  key 
performance  parameters  (KPPs)  are  to  be  derived  from  the  operational  view  of 
the  mission  area  integrated  architecture.  Further,  DoD  is  required  to  develop, 
acquire,  and  deploy  C3I  systems  and  equipment  for  U.S.  Forces  that  are 
compatible  with  existing  and  planned  C3I  systems. 

DoD  Instruction  4630.8.  DoD  Instruction  4630.8  requires  that  joint, 
combined,  coalition,  and  interagency  missions  must  be  supported  through 
interoperable  information  technology  and  national  security  systems  in  global 
operations  across  the  peace-conflict  spectrum.  All  information  technology  and 
national  security  systems  for  U.S.  Forces  use  are  to  be  considered  for  use  by 
joint,  allied,  and  other  U.S.  Government  departments  and  agencies. 

Information  technology  and  national  security  systems’  interoperability  and 


43 


supportability  requirements  are  to  be  identified  through  the  mission  area 
integrated  arehiteetures.  Interoperability  requirements  are  required  to  be 
managed,  evaluated,  and  reported  throughout  the  life  of  the  system.  All  DoD 
aequisition  eategory  programs  will  use  a  C4I  support  plan  to  doeument 
interoperability  requirements.  Additionally,  interoperability  and  supportability 
requirements  are  required  to  be  balaneed  with  the  need  for  information 
assuranee.  The  DoD  Components  are  required  to  submit  mission  need 
statements,  CRDs,  ORDs,  and  C4I  support  plans  for  all  information  teehnology 
and  national  seeurity  system  aequisitions  or  proeurements  to  the  Joint  Staff  for 
review  to  eomply  with  joint  poliey,  doetrine,  and  interoperability  requirements. 
Additionally,  requirements  doeuments  are  to  be  reviewed  by  the  Direetor, 
Defense  Information  Systems  Ageney  for  eomplianee  with  the  DoD  Joint 
Teehnieal  Arehiteeture. 

Furthermore,  the  Joint  Staff  is  to  develop,  approve,  and  direet  the  use  of  the 
Joint  Mission  Area-based  Joint  Operational  Arehiteeture  and  direet  its  use  when 
developing  information  teehnology  and  national  seeurity  systems’ 
interoperability  requirements.  The  Instruetion  states  that  the  Direetor, 
Operational  Test  and  Evaluation  is  to  develop  poliey  and  proeesses  to  test  and 
evaluate  information  teehnology  and  national  seeurity  systems  in  order  to 
aeeurately  assess  the  level  of  interoperability  of  the  tested  system  with  the 
intended  family-of-sy stems  with  whieh  it  must  operate.  Additionally,  the 
Direetor,  Operational  Test  and  Evaluation  is  required  to  eonfirm  that  the  TEMP 
has  identified  information  teehnology  and  national  seeurity  systems’ 
interoperability  requirements  for  programs  tested  under  the  Direetor, 

Operational  Test  and  Evaluation’s  oversight. 

DoD  Instruction  5000.2.  DoD  Instruetion  5000.2  states  that  the  user 
representative,  with  support  from  the  operational  test  and  evaluation  eommunity, 
develops  the  needs  expressed  in  the  mission  need  statement  into  requirements  in 
the  form  of  CRDs,  if  applieable,  and  ORDs.  CRDs  eontain  eapabilities-based 
requirements  to  develop  individual  ORDs  that  provide  a  eommon  framework 
and  operational  eoneept.  The  CRD  is  an  oversight  tool  that  establishes 
overarehing  requirements  for  a  family  of  systems.  Validated  ORDs  translate  the 
mission  needs  statement  and,  if  applieable,  CRDs  into  broad,  flexible,  and 
time-phased  operational  goals  that  are  further  detailed  and  refined  into  speeifie 
operational  eapability  requirements  in  the  final  ORD.  The  appropriate 
requirements  authority  validates  all  mission  needs  statements,  CRDs,  and 
ORDs. 


DoD  Regulation.  DoD  Regulation  5000. 2-R  states  that,  during  the 
requirements  generation  proeess,  users  will  develop  interoperability  KPPs  for  all 
CRDs  and  ORDs.  In  developing  the  ORD,  the  ORD  sponsor  eonsiders  using 
the  produets  deseribed  in  the  I  Surveillanee  and  Reeonnaissanee  Arehiteeture 
Eramework  and  universal  resourees  sueh  as  the  joint  teehnieal  arehiteeture.  The 
joint  operational  arehiteeture  and  the  joint  teehnieal  arehiteeture  serve  as  the 
foundation  for  evolutionary  development  of  those  mission  area  integrated 
arehiteetures.  Mission  area  integrated  arehiteetures  are  to  state  information 
teehnology  and  national  seeurity  systems’  interoperability  requirements  in  a 
family-of-systems  mission  area  eontext.  The  user  is  to  derive  family-of-system 
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information  technology  and  national  security  systems  information  exchange 
requirements  from  the  operational  information  exchange  requirements  of  the 
mission  area  integrated  architecture. 

Further,  the  regulation  requires  the  overarching  integrated  product  team  to 
assess  family-of-system  or  system-of-system  capabilities  for  DoD  systems  within 
mission  areas  in  support  of  mission  area  operational  architectures  developed  by 
the  Joint  Staff  before  milestone  decisions.  Further,  the  ORD  sponsor  is  to 
characterize  information  interoperability,  as  applicable,  within  a  family-of- 
systems,  a  mission  area,  and  a  mission  for  all  information  technology  and 
national  security  systems.  Available  mission  area  integrated  architectures  are  to 
be  used  to  develop  information  technology  and  national  security  systems’ 
interoperability  requirements.  The  regulation  also  states  that  the  joint 
operational  architecture  and  the  joint  technical  architecture  serve  as  the 
foundation  for  evolutionary  development  of  mission  area  integrated 
architectures.  The  implementation  of  the  joint  technical  architecture  is  required 
for  all  new,  or  changes  to  existing,  information  technology  systems,  including 
National  Security  systems. 

The  regulation  also  requires  that  all  major  Defense  acquisition  programs, 
programs  on  the  Office  of  the  Secretary  of  Defense  Test  and  Evaluation 
Oversight  list,  legacy  systems,  and  all  programs  and  systems  that  must  be 
interoperable  with  them  be  subject  to  interoperability  evaluations  throughout 
their  life  cycles  to  validate  their  ability  to  support  mission  accomplishment.  At 
their  discretion,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics;  the  ASD(C3I);  the  Chairman  of  the  Joint  Chiefs  of  Staff  ;  the 
Commander,  USJFCOM;  and  the  Director,  Operational  Test  and  Evaluation  can 
place  programs  and  systems  deemed  to  have  significant  interoperability 
deficiencies  on  the  Interoperability  Watch  List  (Watch  List).  Program  managers 
with  a  program  on  the  Watch  List  must  take  corrective  actions  to  address 
identified  interoperability  deficiencies  to  remove  their  programs  from  the  Watch 
List. 

The  regulation  requires  that,  for  programs  on  the  Watch  List,  program 
managers  will  provide  periodic  updates  of  their  status  towards  correcting 
identified  deficiencies  to  senior  representatives  of  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics;  ASD(C3I);  the  Director, 

Operational  Test  and  Evaluation;  the  USJLCOM;  and  the  Joint  Staff.  The 
program  managers  of  DoD  systems,  or  other  cognizant  officials,  and  the 
responsible  test  organization  (either  developmental  or  operational),  in 
conjunction  with  JITC,  are  to  provide  those  updates.  Those  updates  are  to 
support  an  assessment  on  whether  interoperability  issues  are  being  adequately 
addressed,  and  whether  a  status  change  is  warranted;  for  example,  whether  the 
program  or  system  should  be  removed  from  the  Watch  List,  kept  on  the  Watch 
List,  or  proposed  for  Director,  Operational  Test  and  Evaluation  oversight.  Staff 
members  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics;  the  ASD(C3I);  the  Director,  Operational  Test  and  Evaluation;  the 
USJLCOM;  and  the  Joint  Staff  are  to  prepare  quarterly  reports  summarizing  the 
activities  of  systems  and  programs  on  the  Watch  List. 
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Joint  Staff  Instruction  3170. OIB.  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruetion  3 170. OIB  requires  that  the  Joint  Staff  J-6  eertify  mission  need 
statements,  CRDs,  and  ORDs,  regardless  of  aequisition  eategory,  for 
eonformanee  with  joint  C4I  poliey  and  doetrine,  teehnieal  arehiteetural  integrity, 
and  interoperability  standards.  The  Joint  Staff  J-6  is  to  review  and  eomment  on 
interoperability  KPPs  and  eoordinate  C4I  issues  eoneerning  mission  need 
statements,  eapstone  requirement  doeuments,  and  ORDs  with  the  appropriate 
ageneies. 

The  Joint  Staff  J-6  is  to  forward  for  eoordination  the  C4I  interoperability 
requirements  eertifieation  to  the  Joint  Requirement  Oversight  Couneil  for  major 
DoD  aequisition  programs,  major  automated  information  systems,  and  speeial 
interest  programs  or  to  the  sponsoring  DoD  Component  for  major  systems  and 
below  programs.  Failure  for  those  systems  to  meet  a  KPP  threshold  in  an  ORD 
ean  be  a  reason  for  the  system  seleetion  to  be  reevaluated  or  for  the  program  to 
be  reassessed  or  terminated. 

Joint  Staff  Instruction  6212. OIB.  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruetion  6212. OIB  requires  the  USJFCOM  to  provide  comments  to  the  Joint 
Staff  J-6  on  interoperability  issues  for  all  acquisition  category  programs  to 
ensure  that  each  ORD  contains  interoperability  KPPs  and  information  exchange 
requirements.  Furthermore,  commanders  of  combatant  commands.  Military 
Departments,  and  Defense  agencies  are  to  incorporate  interoperability  testing 
into  their  overall  testing  plans  in  coordination  with  the  Defense  Information 
Systems  Agency  (DISA)  and  Joint  Interoperability  Test  Center  (JITC).  In 
addition,  JITC  is  to  certify  test  results  for  all  interoperability  system  tests. 

DoD  Chief  Information  Officer  Memorandum.  In  DoD  Chief 
Information  Officer  Guidance  and  Policy  Memorandum  No.  8-8001, 

March  31,  2000,  the  DoD  Chief  Information  Officer  authorizes  the  Director, 
Operational  Test  and  Evaluation  to  include  critical  operational  issues  addressing 
interoperability  and  information  assurance  in  the  Global  Information  Grid 
operational  test  and  evaluation. 

Information  Assurance  Policy.  DoD  Regulation  5000. 2-R,  DoD 
Instruction  5200.40,  “DoD  Information  Technology  Security  Certification  and 
Accreditation  Process  (DITSCAP),”  December  30,  1997;  Chairman  of  the  Joint 
Chiefs  of  Staff  Instruction  3170. OIB;  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruction  6510. OIC,  “Information  Assurance  and  Computer  Network 
Defense,”  May  1,  2001;  and  Director,  Operational  Test  and  Evaluation 
memorandum,  “Policy  for  Operational  Test  and  Evaluation  of  Information 
Assurance,”  November  17,  1999,  provide  policy  on  information  assurance. 

DoD  Regulation.  DoD  Regulation  5000. 2-R  requires  program 
managers  to  incorporate  information  assurance  requirements  into  program 
design  functions  to  ensure  availability,  integrity,  authenticity,  confidentiality, 
and  nonrepudiation  of  critical  system  information.  Further,  program  managers 
are  to  manage  information  systems  using  the  best  processes  and  practices  to 
reduce  security  risks,  including  the  risks  to  timely  accreditation. 
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Furthermore,  the  regulation  requires  that  information  assuranee  testing  be 
eondueted  on  information  systems  to  ensure  that  planned  and  implemented 
seeurity  measures  satisfy  ORD  and  System  Seeurity  Authorization  Agreement 
requirements  when  the  system  is  installed  and  operated  in  its  intended 
environment.  The  program  manager,  operational  test  and  evaluation  authority, 
and  designated  approving  authority  are  to  eoordinate  and  determine  the  level  of 
risk  assoeiated  with  operating  the  system  and  the  extent  of  seeurity  testing 
required.  Requirements  to  reeonstitute  or  reeover  information  system 
eapabilities  damaged  by  information  assuranee  threat  agents  are  also  to  be  tested 
during  operational  test  and  evaluation. 

The  regulation  requires  that  all  weapon,  eommand,  eontrol,  eommunieations, 
eomputers,  intelligenee,  surveillanee,  and  reeonnaissanee,  and  information 
programs  that  are  dependent  on  external  information  sourees,  or  that  provide 
information  to  other  DoD  systems,  be  assessed  for  information  assuranee.  The 
level  of  information  assuranee  testing  depends  on  the  system  risk  and 
importanee.  Systems  with  the  highest  importanee  and  risk  are  to  be  subjeet  to 
penetration- type  testing  prior  to  the  beyond  low-rate  initial  produetion  deeision. 
Systems  with  minimal  risk  and  importanee  are  to  be  subjeet  to  normal  National 
Seeurity  Ageney  seeurity  and  developmental  testing,  but  not  subjeet  to  field 
penetration  testing  during  operational  test  and  evaluation. 

DoD  Instruction.  DoD  Instruetion  5200.40  establishes  the  DoD 
Information  Teehnology  Seeurity  Certifieation  and  Aeereditation  Proeess  for 
seeurity  eertifieation  and  aeereditation  of  unelassified  and  elassified  information 
teehnology.  The  Proeess  sets  forth  the  aetivities  and  management  strueture  to 
eertify  and  aeeredit  information  teehnology  systems  that  will  maintain  the 
seeurity  posture  of  the  Defense  Information  Infrastrueture.  The  Proeess 
requires  a  System  Arehiteeture  Analysis,  whieh  requires  the  system  arehiteeture 
to  eomply  with  the  Systems  Seeurity  Authorization  Agreement  arehiteeture 
deseription.  The  interfaees  between  this  and  other  systems  are  to  be  identified 
and  the  interfaees  evaluated  to  assess  their  effeetiveness  in  maintaining  the 
seeurity  posture  of  the  infrastrueture. 

Joint  Staff  Instruction  3170. OIB.  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruetion  3 170. OIB  states  that  information  assuranee  is  required  for  all  DoD 
systems  that  are  used  to  enter,  proeess,  store,  display,  or  transmit  DoD 
information,  regardless  of  elassifieation  or  sensitivity.  Information  assuranee 
requirements  are  to  be  eodeveloped  and  eoevolved  with  those  for  information 
interoperability. 

Joint  Staff  Instruction  6510. OIC.  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruetion  6510.01C  provides  joint  poliey  and  guidanee  for  information 
assuranee  and  eomputer  network  defense  operations.  Information  assuranee  is 
eritieal  to  the  military’s  ability  to  eonduet  warfare  and  is  the  responsibility  of  all 
warfighters.  Beeause  of  the  universal  nature  of  the  Global  Information  Grid,  a 
risk  assumed  by  one  organization,  at  any  organization  level,  ean  be  a  risk 
imposed  on  all  DoD  organizations.  Aeeordingly,  the  requirement  for 
implementing  information  assuranee  is  at  all  organization  levels.  The  primary 
method  of  using  information  assuranee  is  through  the  defense-in-depth  strategy. 
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To  prevent  a  potential  breakdown  of  barriers  and  an  invasion  of  the  innermost 
or  most  valuable  parts  of  the  system,  sueeessive  layers  and  safeguards  must  be 
eonstrueted  at  different  loeations  in  the  system.  Those  different  loeations  are 
expressed  as  loeal  eomputing  networks,  enelave  boundaries,  networks,  and 
supporting  infrastruetures.  Through  a  deliberate  risk  analysis  proeess,  program 
managers  ean  make  effeetive  risk  management  deeisions  to  ensure  that  the  most 
effeetive  defense-in-depth  strategy,  given  the  resourees  available,  is  deployed. 
Additionally,  all  DoD  Components  are  required  to  establish  an  aetive  risk 
management  and  mitigation  program  for  information  and  information-based 
proeesses,  and  information  systems,  sueh  as  eommand,  eontrol, 
eommunieations,  and  eomputer  systems;  weapon  systems;  and  information 
infrastruetures  used  by  military  forees.  The  assoeiated  information  is  to  be 
proteeted  based  on  the  value  of  the  information  eontained  in  the  system  and  the 
risks  assoeiated  with  its  eompromise  or  loss. 

Director,  Operational  Test  and  Evalnation  Memorandnm.  The 

Direetor’s  memorandum  establishes  poliey  for  operational  test  and  evaluation  of 
information  assuranee  for  all  Direetor,  Operational  Test  and  Evaluation 
oversight  programs,  ineluding  weapon  systems,  C4I  surveillanee  and 
reeonnaissanee  systems,  and  information  systems.  The  poliey  applies  to 
aequisition  programs  that  have  not  reaehed  the  produetion  deeision.  The  poliey 
eontains  a  four-step  proeess  to  review  information  assuranee  during  operational 
test  and  evaluation,  as  follows. 

•  Requirements,  Threat,  and  Test  Documentation  Review.  During 
this  step,  the  Direetor,  Operational  Test  and  Evaluation  determines 
the  relevanee  of  information-assuranee  testing  based  on  requirements, 
and  assesses  vulnerabilities  and  the  importanee  of  program  funetions 
and  missions. 

•  Test  Strategy  Development.  Eor  those  programs  that  were  not 
waived  in  the  Step  One  review,  the  Direetor,  Operational  Test  and 
Evaluation  eonduets  a  paper  vulnerability  assessment  as  part  of  the 
normal  test  strategy  development  using  experts,  program  offiee 
personnel,  operational  test  aetivity  representatives,  and  users  to 
define  the  degree  to  whieh  operational  information  assuranee  testing 
is  warranted.  Test  plans  are  not  to  be  approved  unless  they  eontain  a 
well-defined  strategy  for  addressing  information  assuranee  eoneerns, 
adequate  resourees,  and  appropriate  measures  against  whieh  to  test 
stated  requirements.  Those  programs  with  operational  test  and 
evaluation  waivers  for  information  assuranee  are  to  note  the  waiver 
in  their  TEMP  and  test  plan. 

•  Review  of  Information  Assurance  Development  Test  and 
Evaluation  and  Computer  Security  Certification  Results  Before 
Entry  into  Operational  Test  and  Evaluation.  Eor  those  systems 
eonsidered  to  possess  potentially  high  or  unknown  residual 
information  assuranee  risk,  the  operational  test  aetivities  are  to 
examine  development  test  and  evaluation  and  Defense  Information 
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Technology  Security  Certification  Accreditation  Process  data, 
including  any  concurrent  operational  assessments  that  may  have 
already  occurred,  in  judging  the  effectiveness  of  that  system. 

•  Evaluation  of  Information  Assurance  Vulnerabilities  During 
Operational  Test.  Those  programs  that  have  undergone  Steps  One, 
Two,  and  Three  and  are  still  judged  to  have  a  high  or  unknown 
vulnerability  are  to  be  subject  to  field  testing  as  part  of  the  initial 
operational  test  and  evaluation. 
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Appendix  G.  DoD  System  Interoperability 

Requirements  Review  Process 


The  C4I  Surveillance  and  Reconnaissance  Architecture  Framework  documents  how 
architectures  are  created.  Once  an  architecture  is  created,  KPPs  in  the  ORD  are 
defined.  The  following  flow  chart  shows  the  interoperability  certification  review 
process.  The  interoperability  certification  review  process  has  three  stages. 


Stage  One  ® 

(a)  The  Defense  agency  or  Military  Department  enters  the  ORD,  regardless  of 
the  acquisition  category,  into  the  JCPAT. 

(b)  The  acquisition  category  I  ORD  is  forwarded  the  to  the  Force  Structure, 
Resources,  and  Assessment  Directorate  J-8  (Joint  Staff  J-8),  who  staffs  the 
requirement  with  the  members  of  the  Joint  Requirement  Oversight  Council. 


‘'^Stage  lasts  35  days. 
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(c)  The  acquisition  category  II  and  below  ORD  is  staffed  with  the  Command, 
Control,  Communications,  and  Computers  Systems  Directorate  J-6  (Joint 
Staff  J-6). 

(d)  The  Joint  Staff  J-6  staffs  the  acquisition  category  II  and  below  ORD  with 
DISA  and  the  USJFCOM  J-8. 

(e)  The  USJFCOM  J-8  staffs  the  document  with  the  USJFCOM  J-6 

(f)  The  USJFCOM  J-6  directs  subject-matter  experts  review  the  ORD  from  a 
warfighter  perspective  to  verify  that  GIG  requirements  are  in  the  ORD.  The 
USJFCOM  J-6  receives  comments  from  the  subject-matter  expert  and  then 
forwards  those  and  any  additional  comments  back  through  the  chain  to  the 
Joint  Staff  J-6. 

(g)  DISA  staffs  the  ORD  through  the  Plans  and  Policy  Division  which  then 
determines  the  lead  JITC  division.  That  division  assigns  a  specific 
subject-matter  expert  to  review  the  ORD  from  a  technical  perspective.  After 
the  review,  the  Plans  and  Policy  Division  consolidates  comments  on  the  ORD 
from  the  subject-matter  expert  with  those  from  other  JITC  divisions. 

(h)  JITC  subject-matter  experts  use  a  checklist  to  ensure  that  their  mission  need 
statements,  CRDs,  ORDs,  C4I  support  plans,  and  TEMPs  meet  DoD  policy 
requirements.  Based  on  the  results  of  the  checklist  review,  comments  are 
entered  in  the  JITC  database  and  forwarded  to  the  Joint  Staff  J-6  through  the 
DISA.  The  JITC  database  maintains  historical  data  on  documentation  for 
DoD  systems  until  operational  testing  is  completed.  JITC  uses  the  data  in  its 
database  and  information  from  other  sources  such  as  symposiums,  field 
demonstrations,  or  program  managers  to  identify  interoperability  and 
information  assurance  requirements. 

Stage  Two^" 

(i)  If  the  Joint  Staff  J-6  does  not  certify  the  ORD  in  stage  one,  the  ORD,  with 
comments,  is  returned  to  the  Defense  agency  or  Military  Department  for 
resolution  of  interoperability  issues.  After  the  issues  identified  in  stage  one 
are  resolved,  stage  two,  which  is  the  same  process  as  stage  one,  begins.  If  no 
deficiencies  are  found  by  the  Joint  Staff  J-6,  the  ORD  may  be  certified  for 
interoperability. 

(j)  After  certification  for  interoperability,  the  Joint  Staff  J-6  returns  the  ORD  to 
the  Military  Department  or  the  Defense  agency,  as  applicable.  The  Military 
Department  or  the  Defense  agency  is  responsible  for  ORD  approval. 


‘’Stage  two  lasts  21  days. 
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(k)  When  the  ORD  is  approved,  the  requirement  eompetes  for  funding  in  the 
Planning,  Programming,  and  Budgeting  System  proeess.  However,  no 
requirement  exists  for  Military  Departments  or  Defense  ageneies  to  submit 
the  final  approved  ORD  to  be  updated  in  the  JCPAT. 

Stage  Three^*" 


Stage  III  is  the  posting  of  the  Aequisition  Category  II  or  III  Milestone  Deeision 
Authority  approved  mission  need  statements,  CRD,  or  ORD.  Approved 
doeuments  are  filed  in  the  JCPAT  with  the  Joint  Staff  J-6  eertifieation  letter. 


‘®Stage  three  suspense  is  15  days  after  the  Joint  Requirements  Oversight  Council  or  the  milestone 
decision  authority  approves  the  mission  need  statement,  CRD,  or  ORD  and  posting  the  document  into  the 
JCPAT,  according  to  the  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  6212. OIB. 
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Appendix  H.  Organizations  with  Access  to  the 

Joint  Command,  Control, 
Communications,  Computers,  and 
Intelligence  Program  Assessment 
Tool 


Office  of  the  Secretary  of  Defense 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and  Computers) 
Inspector  General  of  the  Department  of  Defense 


Joint  Staff 

Director  for  Intelligence  (J-2) 

Director  for  Operations  (J-3) 

Director  for  Logistics  (J-4) 

Director  for  Strategic  Plans  and  Policy  (J-5) 

Director  for  Command,  Control,  Communications,  and  Computers  (J-6) 
Director  for  Operational  Plans  and  Interoperability  (J-7) 

Director  for  Force  Structure,  Resources,  and  Assessment  (J-8) 


Department  of  the  Army 

Deputy  Chief  of  Staff  Operations  and  Plans,  Headquarters  Department  of  the  Army 
Commander,  Communications  and  Electronics  Command 
Commander,  Executive  Agent  Theater  Joint  Tactical  Networks 
Commander,  Training  and  Doctrine  Command 


Department  of  the  Navy 

Deputy  Assistant  Secretary  of  the  Navy 

Deputy  Chief  of  Naval  Operations  (Resources,  Requirements,  and  Assessments) 
Director,  Equipment  and  Requirements  Division,  United  States  Marine  Corps 


Department  of  the  Air  Force 

Deputy  Chief  of  Staff  for  Air  and  Space  Operations 
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Unified  Commands 


Commander,  U.S.  European  Command 
Commander,  U.S.  Paeifie  Command 
Commander,  U.S.  Joint  Forees  Command 
Commander,  U.S.  Southern  Command 
Commander,  U.S.  Central  Command 
Commander,  U.S.  Spaee  Command 

North  Ameriean  Aerospaee  Defense  Command 
Commander,  U.S.  Speeial  Operations  Command 
Commander,  U.S.  Strategie  Command 
Commander,  U.S.  Transportation  Command 


Other  Defense  Organizations 

Direetor,  Operational  Test  and  Evaluation 
Direetor,  Defense  Finanee  and  Aeeounting  Serviee 
Direetor,  Defense  Intelligenee  Ageney 
Direetor,  Defense  Logisties  Ageney 
Direetor,  Defense  Information  Systems  Ageney 
Direetor  for  Operations 

Direetor  for  Aequisitions,  Logisties,  and  Faeilities 
Direetor  for  Strategie  Plans  and  Poliey 
Direetor  for  Engineering  and  Information 
Direetor  for  Joint  Requirement  Analysis  and  Integration 
Direetor  for  Command,  Control,  Communieations,  Computers  and  Intelligenee 
Modeling,  Simulation,  and  Assessment 
Direetor  for  Applieations  Engineering  Direetorate 
Program  Planning  Offiee 
Direetor  for  Interoperability 
Prineipal  Direetor  for  Network  Serviees 
Commander,  Joint  Interoperability  Test  Command 
Commander,  Joint  Speetrum  Center 
Commander,  Joint  Theater  Air  and  Missile  Defense  Organization 
Direetor,  Joint  Warfighting  Capabilities  and  Assessment 
Commander,  Missile  Defense  Ageney 
Direetor,  National  Imagery  and  Mapping  Ageney 
Direetor,  National  Reeonnaissanee  Offiee 
Direetor,  National  Seeurity  Ageney 
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Appendix  I.  Response  to  Office  of  the  Secretary 

of  Defense  and  Defense  Agency 
Comments  Concerning  the  Report 

Our  detailed  response  to  the  eomments  from  the  Direetor,  Interoperability, 
Offiee  of  the  Under  Seeretary  of  Defense  for  Aequisition,  Teehnology,  and 
Logisties;  the  Direetor,  Arehiteeture  and  Interoperability,  Offiee  of  the 
ASD(C3I);  the  Inspeetor  General,  Offiee  of  the  Commander,  USJFCOM;  the 
Inspeetor  General,  DISA;  the  Direetor,  Joint  Staff;  and  the  Co-Chair, 
Interoperability  Senior  Review  Panel,  on  statements  in  the  draft  report  follow. 
The  eomplete  text  of  those  comments  is  in  the  Management  Comments  section 
of  this  report. 


Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  Comments  on  Finding  and  Audit  Response 


The  Director,  Interoperability,  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics,  commented  on  the  implementation  of 
information  assurance  and  the  term  “combat  developers. 

Implementation  of  Information  Assurance.  The  Director  stated  that  the 
Conclusion  section  of  the  report  states  that,  “...information  assurance  must  be 
implemented  whenever  a  system  is  deemed  interoperable...,”  and  that  the 
statement  is  offered  as  a  given.  However,  he  believed  that  the  statement  may 
tend  to  imply  equality  of  the  two  related  measures.  The  Director  stated  that  an 
appropriate  qualification  might  be,  “Information  assurance  must  be  implemented 
appropriately  and  commensurate  to  the  level  of  interoperability  required.  ” 

Audit  Response.  We  revised  the  report  as  suggested. 

Combat  Developers.  The  Director  stated  that  the  term  “combat  developers” 
was  used  in  the  Conclusion  section  of  the  report  without  defining  it. 

Audit  Response.  The  term  “combat  developers”  was  defined  in  Appendix  B, 
“Definitions  of  Technical  Terms,”  of  the  report.  To  clarify  the  Conclusion 
section,  we  added  a  footnote  to  also  define  the  term,  which  refers  to  the 
requirements  generation  process. 


Assistant  Secretary  of  Defense  (Command,  Control, 

Communications,  and  Intelligence)  Comments  on  Finding 


The  Director,  Architecture  and  Interoperability,  Office  of  the  ASD(C3I) 
suggested  some  editorial  changes  that  we  considered  and  made  where  deemed 
appropriate. 
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U.S.  Joint  Forces  Command  Comments  on  Finding  and  Audit 
Response 


The  Inspector  General,  USJFCOM  commented  on  analyzing  interoperability  and 
information  assurance,  conducting  analytical  or  quantitative  modeling  and 
simulation,  analyzing  ORD  requirements,  comparing  the  proposed  system  ORD 
against  the  ORDs  of  the  systems  with  which  it  must  interoperate,  providing  a 
joint  perspective  concerning  interoperability  and  integration,  and  comparing 
requirement  documents  of  proposed  DoD  systems  within  joint  mission  area 
architectures. 

Interoperability  and  Information  Assurance  Analysis.  The  Inspector  General 
stated  that  the  paragraph  stating  that  “The  Commander,  USJFCOM  did  not 
analyze  interoperability  and  information  assurance  requirements  in  the 
operational  and  system  architectures  for  DoD  systems  as  part  of  the 
interoperability  certification  process  for  ORDs”  does  not  accurately  reflect  the 
requirements  review  process  at  the  USJFCOM  and  the  required  actions  as 
directed  by  Chairman  of  the  Joint  Chiefs  of  Staff  Instructions  3170.01B  and 
6212. OIB.  Further,  USJFCOM  does  analyze  interoperability  and  information 
assurance.  Without  overarching  joint  mission  area  architectures,  USJFCOM 
analyzes  integration  requirements  in  coordination  with  Chairman  of  the  Joint 
Chiefs  of  Staff  Instructions  3170. OIB  and  6212. OIB  from  known  and  existing 
systems  as  they  integrate  with  existing  CRDs. 

Audit  Response.  As  stated  in  the  finding,  the  analysis  of  interoperability  and 
information  assurance  requirements  that  the  USJFCOM  subject-matter  experts 
performed  for  proposed  DoD  system’s  requirements  should  be  made  to  ORD 
requirements  for  other  systems  that  the  proposed  system  will  be  interoperable 
with  in  their  intended  mission  architectures  in  addition  to  those  requirements 
contained  in  existing  CRDs. 

Analytical  or  Quantitative  Modeling  and  Simulation.  The  Inspector  General 
stated  that  USJFCOM  did  not  compare  ORDs  to  joint  mission  architectures 
before  system  development  started  because  it  did  not  have  the  resources  to 
conduct  that  level  of  review.  Although  USJFCOM  strongly  disagrees  that  the 
depth  of  the  ORD  review  as  outlined  in  the  report  is  a  USJFCOM  mission, 
USJFCOM  would  like  to  be  able  to  conduct  the  reviews  as  outlined  in  the 
report,  including  detailed  analysis. 

Audit  Response.  In  Section  922,  Public  Law  105-261,  Congress  expressed  its 
sense  that  the  commander  of  a  combatant  command  should  be  provided  with 
appropriate  and  sufficient  resources  for  joint  warfighting  experimentation.  It 
listed  the  responsibilities  and  authorities  that  should  accompany  that  designation, 
including  improving  interoperability  and  making  recommendations  to  the 
Chairman  of  the  Joint  Chiefs  of  Staff  on  mission  needs  statements  and  ORDs. 
Accordingly,  if  USJFCOM  believes  that  resources  are  insufficient,  it  should 
request  the  resources  needed  to  fulfill  the  sense  of  Congress. 
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ORD  Requirements.  The  Inspector  General  stated  that  the  requirement  to 
review  ORDs  in  context  with  the  existing  family  of  systems  is  not  a  USJFCOM 
requirement.  Further,  the  ORD  review  process,  as  outlined  in  Chairman  of  the 
Joint  Chiefs  of  Staff  Instructions  3170. OIB  and  6212. OIB,  was  to  ensure  that  the 
new  ORDs  were  interoperable  with  the  future  family  of  systems  as  outlined  or 
described  in  the  CRDs.  The  CRD  details  the  future  system  requirements  for  the 
family  of  systems  that  include  the  current  and  potential  innovation  resulting 
from  technology  to  be  added.  USJFCOM  works  with  the  ORD  authors  to 
ensure  that  the  new  ORDs  are  compliant  with  the  CRDs.  Further,  the  role  of 
the  CRDs  is  to  serve  as  the  road  map  to  future  interoperability,  which  is  why 
Chairman  of  the  Joint  Chiefs  of  Staff  Instructions  3170. OIB  and  6212. OIB 
require  the  traceability  of  ORD  requirements  to  CRD  requirements  and  not  to 
the  current  or  existing  systems  within  the  particular  family  of  systems.  In 
addition,  USJFCOM  does  realize  that  additional  efforts  are  required  to  ensure 
interoperability  and,  with  additional  resources,  would  want  to  participate  in 
those  efforts.  Additionally,  DoD  lacks  a  reliable  repository  of  all  existing 
family  of  systems  information. 

Audit  Response.  USJFCOM  had  not  documented  that  it  lacked; 

•  resources  needed  to  fulfill  its  responsibility  for  assessing 
interoperability  requirements  for  proposed  DoD  weapon  systems  or 

•  a  reliable  repository  of  all  existing  family  of  systems  information. 

USJFCOM  should  be  proactive  in  acquiring  additional  resources  to  ensure  the 
interoperability  of  systems  and  to  establish  a  reliable  repository  of  all  existing 
family  of  systems  information,  as  specified  in  Section  922,  Public  Law  105-261, 
discussed  above. 

Proposed  System  ORD  Comparison.  The  Inspector  General  stated  that 
subject-matter  experts  were  not  required  or  authorized  to  compare  the  proposed 
system  ORD  against  the  ORDs  of  the  systems  with  which  it  must  interoperate. 
Further,  the  development  of  ORDs  was  the  excusive  role  of  the  Military 
Departments  and  Defense  agencies  and,  even  though  USJFCOM  did  have  a  role 
in  the  review  process,  approval  was  the  responsibility  of  the  Joint  Requirements 
Oversight  Council.  The  report  misinterpreted  what  the  actual  role  of 
USJFCOM  was  in  the  process  and  grossly  overstated  the  requirements  for 
USJFCOM.  However,  USJFCOM  did  concur  that,  with  additional  resources,  it 
could  improve  the  interoperability  effort  in  many  areas;  however,  those  areas 
were  not  its  assigned  tasks  and  were  unfunded. 

Audit  Response.  The  USJFCOM  has  the  authority  to  obtain  additional 
resources  to  improve  interoperability  throughout  DoD.  The  Joint  Staffs 
“Unified  Command  Plan  1999,”  emphasizes  the  role  of  USJFCOM  as  the  chief 
advocate  of  jointness  and  the  importance  of  enhancing  jointness  and 
interoperability  throughout  DoD.  As  part  of  its  role  in  transforming  U.S. 

Armed  Forces  to  meet  the  security  challenges  of  the  2U'  century,  USJFCOM 
has  the  responsibility  to  support  the  development  and  integration  of  fully 
interoperable  systems  and  capabilities,  including  C4I  surveillance  and 


57 


reconnaissance.  Further,  the  sense  of  Congress,  as  expressed  in  Section  922, 
Public  Law  105-261  encourages  a  combatant  commander,  such  as  USJFCOM, 
to  be  proactive  in  obtaining  appropriate  and  sufficient  resources  to  fulfill  its 
responsibility  for  assessing  interoperability  requirements  of  proposed  DoD 
weapon  systems  to  enable  the  warfighter  to  have  interoperable  systems. 

Joint  Perspective.  The  Inspector  General  suggested  that  another 
recommendation  be  added  to  Recommendation  4.  Specifically,  he  proposed 
that  the  Director,  Command,  Control,  Communications,  and  Computers 
Systems  Directorate  (J-6),  Office  of  the  Chairman  of  the  Joint  Chiefs  of  Staff: 

Provide  a  joint  perspective  as  the  Chairman’s  advocate  for 
interoperability  and  integration,  to  team  with  the  Joint  Warfighting 
Capability  Assessments  (JWCAs)  in  developing  Joint  Mission  Area 
Architectures. 

The  Inspector  General  suggested  the  new  recommendation  because  joint  mission 
area  architectures  are  key  to  determining  the  usefulness  of  systems  within  DoD. 
Further,  as  the  warfighters  advocate  for  interoperability  and  integration, 
USJFCOM  should  participate  in  the  architecture  process  to  ensure  that 
warfighter  needs  and  interests  are  met. 

Audit  Response.  The  report  did  not  address  the  merits  of  a  joint  perspective 
interaction  with  joint  warfighting  capability  assessments  when  developing  joint 
mission  area  architectures.  However,  we  will  consider  addressing  that  joint 
perspective  issue  when  reviewing  interoperability  and  information  assurance 
policies  at  the  unified  command  level,  in  a  subsequent  audit. 

Comparison  of  Requirement  Documents.  The  Inspector  General  suggested 
that  another  recommendation  be  added  to  Recommendation  4.  Specifically,  he 
proposed  that  the  Director,  Command,  Control,  Communications,  and 
Computers  Systems  Directorate  (J-6),  Office  of  the  Chairman  of  the  Joint  Chiefs 
of  Staff: 


Compare  requirement  documents  of  proposed  DoD  systems  within 
JMA  [joint  mission  area]  Architectures,  as  they  are  completed, 
against  the  other  requirement  documents  in  their  related  Joint  Mission 
Area  Architecture  to  verily  the  completeness  of  stated  interoperability 
requirements. 

The  Inspector  General  provided  the  same  rationale  for  this  recommendation  as 
he  gave  for  his  recommendation  to  provide  a  joint  perspective,  as  discussed 
above. 

Audit  Response.  The  report  did  not  specifically  address  the  comparison  of 
requirement  documents  of  proposed  DoD  systems  within  joint  mission  area 
architectures  against  the  other  requirement  documents  in  their  related  joint 
mission  area  architecture(s).  However,  we  will  consider  addressing  that 
comparison  issue  when  reviewing  interoperability  and  information  assurance 
policies  at  the  unified  command  level,  in  a  subsequent  audit. 
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Defense  Information  Systems  Agency  Comments  on  Finding 
and  Audit  Response 


The  Inspector  General,  Defense  Information  Systems  Agency  commented  on 
DoD  interoperability  policy,  the  Interoperability  Watch  List,  inconsistent  DoD 
policy,  conformance  with  Joint  Chiefs  of  Staff  policy,  interoperability  analysis, 
access  to  the  JCPAT  database,  and  interoperability  certification.  In  addition, 
the  Inspector  General  suggested  some  editorial  changes  that  we  considered  and 
made  where  deemed  appropriate. 

DoD  Interoperability  Policy.  The  Inspector  General  stated  that  DoD 
Directive  4630.5  and  DoD  Instruction  4630.8  establish  DoD  interoperability 
policy  and  that; 

•  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  implemented  its  interoperability 
responsibilities  in  DoD  Directive  4630.5  and  DoD 
Regulation  5000. 2-R,  and 

•  the  Joint  Staff  implemented  its  interoperability  responsibilities  in 
Chairman  of  the  Joint  Chiefs  of  Staff  Instructions  3170.01B  and 
6212. OIB. 

Further,  while  ASD(C3I)  was  revising  DoD  Directive  4630.5,  the  Joint  Staff 
rewrote  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  62 12.0 IB,  which 
incorporated  the  intended  changes  to  DoD  Directive  4630.5  before  its  release  in 
January  2002.  In  addition,  the  ASD(C3I),  as  a  staff  element  of  the  Office  of  the 
Secretary  of  Defense,  should  be  a  higher  authority  than  the  Joint  Chiefs  of  Staff. 
Therefore,  the  Inspector  General  stated  that  the  report  criticism  could  be  that 
DoD  Directive  4630.5  and  DoD  Instruction  4630.8  were  outdated  because  of 
developments  in  architecture  frameworks  and  acquisition  documents. 
Alternatively,  one  could  say,  with  equal  validity,  that  Chairman  of  the  Joint 
Chiefs  of  Staff  Instruction  62 12.0 IB: 

•  was  prematurely  issued  because  it  was  in  conflict  with  DoD 
Directive  4630.5  and  DoD  Instruction  4630.8  and 

•  lacked  the  authority  of  DoD  Directive  4630.5  and  DoD 
Instruction  4630.8  for  the  changes. 

Audit  Response.  The  Joint  Chiefs  of  Staff  were  proactive.  The  Office  of  the 
ASD(C3I)  guidance  had  not  been  updated  for  10  years.  As  the  Inspector 
General  stated,  DoD  Directive  4630.5  and  DoD  Instruction  4630.8  were 
partially  outdated  because  of  developments  in  architecture  frameworks  and 
acquisition  documents. 

Interoperability  Watch  List.  The  Inspector  General  stated  that  DoD 
Instruction  4630.8  did  not  specifically  require  the  Director,  Operational  Test 
and  Evaluation  to  establish  criteria  for  placing  DoD  weapon  systems  in  the 
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development  phase  of  the  aequisition  proeess  on  the  Interoperability  Wateh  List. 
Further,  the  Interoperability  Senior  Review  Panel  was  addressing  the  proeess  for 
the  Interoperability  Wateh  List  and  had  identified  a  proeess  for  nominating 
systems  to  the  list.  The  Inspeetor  General  also  stated  that  systems  had  been 
nominated  for  the  Interoperability  Wateh  List,  but  none  were  aetually  put  on  the 
List  to  date. 

Audit  Response.  As  evideneed  in  the  report,  without  speeifie  eriteria  for 
plaeing  DoD  weapon  systems  on  the  Interoperability  Wateh  List,  the  Offiee  of 
the  Seeretary  of  Defense,  the  Joint  Staff,  the  Defense  ageneies,  and  the  Military 
Departments  have  not  plaeed  any  DoD  weapon  systems  on  the  List.  This 
oeeurred  even  though  the  Direetor,  Operational  Test  and  Evaluation  eited 
interoperability  testing  eoneerns  for  21  DoD  systems  in  the  FY  2001  Annual 
Test  Report. 

Inconsistent  DoD  Policy.  The  Inspeetor  General  stated  that  the  Defense 
Information  Systems  Ageney  did  not  believe  that  the  DoD  poliey  is  ineonsistent, 
but  rather  that  the  release  dates  for  revision  of  the  various  doeuments  were  not 
as  eoordinated  as  possible.  Further,  the  eontents  of  the  various  doeuments  were 
eoordinated  and  are  now  synehronized  and  undergoing  another  review  and 
update  eyele. 

Audit  Response.  Whether  or  not  the  release  dates  for  revision  of  the  various 
doeuments  were  eoordinated,  DoD  Direetive  4630.5  and  DoD 
Instruetion  4630.8  were  outdated  beeause  of  developments  in  arehiteeture 
frameworks  and  aequisition  doeuments  over  the  10-year  period  between 
doeument  updates. 

Conformance  with  Joint  Chiefs  of  Staff  Policy.  The  Inspeetor  General  stated 
that  Joint  Chiefs  of  Staff  poliey  and  instruetion  should  eonform  to  ASD(C3I) 
guidanee,  not  the  other  way  around. 

Audit  Response.  We  agree;  however,  the  Joint  Staff  was  proaetive  in 
implementing  agreed-upon  poliey  needed  beeause  of  developments  in 
arehiteeture  frameworks  and  aequisition  doeuments. 

Interoperability  Analysis.  The  Inspeetor  General  reeommended  revising  the 
report’s  statements  about  the  interoperability  proeess  beeause  the  ORD  review 
proeess  requires  the  originator  to  identify  a  proposed  system’s  interoperability 
requirements  in  the  eontext  of  the  other  systems  with  whieh  it  must  interoperate. 

Audit  Response.  Combat  developers  have  neither  a  eomplete  database  of  all 
DoD  requirement  and  aequisition  doeuments  nor  aeeess  to  all  joint  mission  area 
arehiteetures  to  use  as  a  referenee  when  developing  ORD  interoperability 
requirements.  Aeeordingly,  USJFCOM  has  an  important  role  in  ensuring  that 
interoperability  requirements  are  fully  defined  in  the  ORD  review  proeess. 

Access  to  JCPAT  Database.  The  Inspeetor  General  stated  that  the 

third  sentenee  under  the  subheading,  “Aeeess  to  Database,”  indieated  that  of 

756  users,  15  were  eontraetor  support  personnel,  and  that  the  fourth  sentenee 
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indicated  that  DISA  could  identify  the  total  number  of  users  but  could  not 
distinguish  DoD  from  contractor  personnel.  He  concluded  that  both  statements 
could  not  be  correct. 

Audit  Response.  The  Inspector  General  was  citing  statements  from  the 
discussion  draft  of  this  report,  not  the  draft  report.  We  revised  those  statements 
before  issuing  the  draft  report. 

Interoperability  Certification.  The  Inspector  General  stated  that  the  phrase, 
“interoperability  certification,”  is  ambiguous.  Further,  he  stated  that  the  Joint 
Staff  issues  interoperability  certifications  of  requirements  and  supportability 
documents,  which  are  two  separate  types  of  certifications,  and  issues  system 
validation  memoranda.  He  also  stated  that  JITC  issues  system  interoperability 
test  certifications  based  on  interoperability  testing. 

Audit  Response.  Page  4,  Footnote  2,  of  the  report  discussed  the  two  separate 
types  of  interoperability  certifications.  In  response  to  the  management 
comments,  we  added  the  term,  “Interoperability  Certification  Process,”  to 
Appendix  B,  “Definitions  of  Technical  Terms.” 


Joint  Staff  Comments  on  Finding  and  Audit  Response 


The  Director,  Joint  Staff  commented  on  analyzing  interoperability  and 
information  assurance  requirements;  updating  the  Joint  Command,  Control, 
Communications,  Computers,  and  Intelligence  Program  Assessment  Tool 
(JCPAT);  conducting  analytical  or  quantitative  modeling  and  simulation; 
analyzing  ORD  requirements;  comparing  the  proposed  system  ORD  against  the 
ORDs  of  the  system  with  which  it  must  interoperate;  developing  the  Automated 
Commander-in-Chief  Integrated  System  Tool;  and  maintaining  the  database.  In 
addition,  the  Director  suggested  some  editorial  changes  that  we  considered  and 
made  where  deemed  appropriate. 

Interoperability  and  Information  Assurance  Analysis.  The  Director  provided 
comments  similar  to  those  made  by  the  Inspector  General,  USJFCOM,  for 
which  we  previously  provided  an  audit  response. 

JCPAT.  The  Director  stated  that  the  USJFCOM  did  not  update  the  JCPAT 
database  to  include  all  interoperability  certifications,  TEMPs,  and  C4I  support 
plans  for  DoD  systems  because  those  documents  were  in  hard  copy  form  and, 
therefore,  impede  interoperability  certification.  Further,  the  Director  stated  that 
the  Joint  Staff  J-6  maintains  a  hard  copy  of  programs  and  documents  that 
predate  the  JCPAT. 

Audit  Response.  As  indicated  in  the  finding,  the  older  documents  in  the 
JCPAT  should  be  archived  and  all  applicable  documents  should  be  incorporated 
into  the  JCPAT.  Archiving  and  incorporating  documents  is  important  because 
other  organizations  outside  the  Joint  Staff  use  the  JCPAT  to  ensure  that  all 
information  exchange  requirements  are  encompassed  in  ORDs.  Therefore,  the 
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JCPAT  data  need  to  be  eomplete  and  aeeessible.  The  inelusion  of  C4I  support 
plans  and  TEMPs  in  the  JCPAT  database  would  help  eombat  developers  to  more 
fiilly  review  and  determine  system  interoperability  requirements. 

Analytical  or  Quantitative  Modeling  and  Simulation.  The  Direetor  provided 
eomments  similar  to  those  made  by  the  Inspeetor  General,  USJFCOM,  for 
whieh  we  previously  provided  an  audit  response. 

ORD  Requirements.  The  Direetor  provided  comments  similar  to  those  made 
by  the  Inspector  General,  USJFCOM,  for  which  we  previously  provided  an 
audit  response. 

Proposed  System  ORD  Comparison.  The  Director  provided  comments  similar 
to  those  made  by  the  Inspector  General,  USJFCOM,  for  which  we  previously 
provided  an  audit  response. 

Automated  Commander-in-Chief  Integrated  System  Tool.  The  Director 
stated  that  the  recommended  Automated  Commander-in-Chief  Integrated  System 
Tool  was  under  development,  and  DoD  had  not  accepted,  approved,  or  certified 
it.  Further,  he  stated  that  USJFCOM  could  evaluate  the  usefulness  of  this  or 
any  other  tool  that  USJFCOM  uses  in  the  requirements  process. 

Audit  Response.  The  report  discusses  the  Automated  Commander-in-Chief 
Integrated  System  Tool  only  as  a  possible  automated  process  to  assist 
USJFCOM  in  evaluating  interoperability  requirements  in  proposed  ORDs. 

Maintenance  of  the  Database.  The  Director  suggested  revising  the  sentence 
that  discusses  searching  for  interoperability  certification  documents  in  the 
JCPAT  database.  He  stated  that  the  reason  for  the  revision  was  because 
requirements  documents  were  submitted  at  each  milestone.  Further,  the 
Director  stated  that  depending  upon  the  milestone,  a  particular  program  may 
have  more  than  one  document.  He  also  stated  that  a  reviewer  would  have  to  be 
aware  of  the  milestone  and  the  date  processed  to  determine  the  current 
document.  However,  the  Director  stated  that  the  Joint  Staff  agreed  that  the  user 
could  not  be  assured  that  the  document  is  the  most  recently  approved  or 
validated  document  for  Acquisition  Category  II  and  below  documents.  Further, 
he  stated  that,  because  the  Joint  Requirements  Oversight  Council  was  a 
validating  authority,  the  Director,  Force  Structure,  Resources,  and  Assessment 
(J-8)  post-validated  Acquisition  Category  I  or  Joint  Staff  Issue  programs  to  the 
J-8  tool.  The  Director  also  stated  that  the  validated  documents  are  available  in 
both  the  J-6  and  J-8  tools.  In  conclusion,  he  stated  that  recent  changes  to  the  J-6 
JCPAT  should  alleviate  the  confusion  concerning  the  availability  of  up-to-date 
program  documentation  in  the  JCPAT  database. 

Audit  Response.  Even  though  the  Director  stated  that  the  validated  documents 
are  available  in  the  J-6  and  J-8  tools  and  that  recent  changes  to  the  J-6  JCPAT 
should  alleviate  the  confusion  concerning  the  availability  of  up-to-date  program 
documentation,  the  user  needs  assurance  that  the  document  located  in  the 
JCPAT  database  is  the  most  recently  certified  document.  Therefore,  DISA 
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needs  to  arehive  outdated  versions  of  doeuments  so  that  users  ean  loeate  and 
obtain  the  most  up-to-date  approved  doeuments  for  making  interoperability 
determinations  for  DoD  systems. 

Interoperability  Senior  Review  Panel  Comments  on  Finding 
and  Audit  Response 

The  Co-Chair,  Interoperability  Senior  Review  Panel  stated  that  the 
Interoperability  Senior  Review  Panel  eonsists  of  senior  leaders  from  the  Offiees 
of  the  Under  Seeretary  of  Defense  for  Aequisition,  Teehnology,  and  Logisties; 
the  ASD(C3I);  the  Joint  Staff;  U.S.  Joint  Forees  Command;  the  Direetor  for 
Programs,  Analysis,  and  Evaluation;  and  the  Direetor,  Operational  Test  and 
Evaluation.  He  reeognized  that  the  primary  objeetive  of  this  report  was  to 
evaluate  the  Offiee  of  the  Seeretary  of  Defense  and  the  Defense  ageneies’ 
implementation  of  DoD  interoperability  and  information  assuranee  polieies,  but 
he  stated  that  the  report  did  not  address  the  issue  that  DoD  is  not  struetured  to 
organize,  train,  and  equip  required  joint  eapabilities.  The  Co-Chair  also  stated 
that  the  Military  Departments  determine  requirements  and  then  resouree  those 
requirements  based  on  their  individual  priorities  instead  of  joint  priorities. 
Further,  the  Co-Chair  stated  that  interoperability  and  information  assuranee  for 
information  teehnology  and  national  seeurity  systems  should  be  viewed  in  the 
wider  eontext  of  doetrine,  organization,  training,  material,  leadership 
development,  personnel,  and  faeilities  instead  of  only  system  of  systems  eontext 
or  family  of  systems  eontext,  or  both.  He  eoneluded  that  joint  interoperability 
requires  a  synehronized  effort  and  resourees  aeross  the  entire  doetrine, 
organization,  training,  material,  leadership  development,  personnel,  and 
faeilities  speetrum. 

Audit  Response.  As  the  Co-Chair  noted,  the  report  did  not  address  whether 
DoD  is  struetured  to  organize,  train,  and  equip  required  joint  eapabilities. 
However,  we  will  eonsider  addressing  that  strueturing  issue  in  a  subsequent 
audit  when  we  review  interoperability  and  information  assuranee  polieies  at  the 
unified  eommand  level.  Beeause  eertain  recommendations  required  a 
synchronized  effort  and  resources,  we  redirected  those  recommendations  to  the 
Co-Chair  to  coordinate  corrective  actions  to  implement  policies. 
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Appendix  J.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Deputy  Under  Secretary  of  Defense  (Acquisition  Initiatives) 

Under  Secretary  of  Defense  (Comptroller)/Chief  Financial  Officer 
Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 
Director,  Operational  Test  and  Evaluation 


Joint  Staff 

Director,  Joint  Staff 

Director  for  Command,  Control,  Communications,  and  Computers  Systems  (J-6) 
Director  for  Force  Structure,  Resources,  and  Assessment  (J-8) 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Acquisition,  Logistics,  and  Technology) 
Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Unified  Command 

Commander,  U.S.  Joint  Forces  Command 


Other  Defense  Organizations 

Director,  Defense  Information  Systems  Agency 
Commander,  Joint  Interoperability  Test  Command 
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Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 

Offiee  of  Information  and  Regulatory  Affairs 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subeommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Serviees 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subeommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Serviees 
House  Committee  on  Government  Reform 

House  Subeommittee  on  Government  Effieieney,  Finaneial  Management,  and 
Intergovernmental  Relations,  Committee  on  Government  Reform 
House  Subeommittee  on  National  Seeurity,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subeommittee  on  Teehnology  and  Proeurement  Poliey,  Committee  on 
Government  Reform 
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Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  Comments 


Final  Report 
Reference 


ACQUISITION, 
TECHNOLOGY 
AND  LOGISTICS 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3000 


August  14,  2002 


MEMORANDUM  FOR  DOD  INSPECTOR  GENERAL 

ATTN:  AUDIT  FOLLOW-UP 

THROUGH:  Director.  Acquisition  Resources  and  Analysis  30^0  >l- 

SUBJECT:  Draft  Audit  Report  No.  D2002AE-0009  “Implementation  of  Interoperability  and 

Information  Assurance  Policies  for  Acquisition  of  DoD  Weapons  Systems”,  July  19, 
2002 

This  is  in  response  to  your  memorandum  dated  July  19,  2002,  requesting  comments  on  the  subject 
audit  report. 

•  Conclusions,  Page  15,  Line  7: 

“. .  .information  assurance  must  be  implemented  whenever  a  system  is  deemed 
interoperable....” 

-  is  offered  as  a  given;  however,  as  slated  without  appropriate  qualification, 
may  tend  to  imply  equality  of  the  two  related  measures. 

-  an  appropriate  qualification  might  be,  “Information  assurance  must  be 
implemented  appropriately  and  commensurate  to  the  level  of 
interoperability  required.” 


•  Conclusions,  Page  15,  Line  1 1 

You  have  used  the  term  “combat  developers”  without  defining  it.  If  you  are 
referring  to  program  managers,  then  the  reference  is  incorrect.  PMs  should  not 
interpret  requirements.  They  are  set  by  the  Requirements  Generation  Process.  If 
you  arc  referring  to  the  requirements  groups  in  the  services  or  Joint  Staff,  then  the 
activity  is  already  being  accomplished. 

•  Recommendation  1,  Page  16: 

The  wording  “in  coordination  with”  can  be  construed  to  give  the  ASD(C3I)  primacy  over 
AT&L  and  the  Joint  Staff  in  integrating  revisions  to  all  documents  related  to 
interoperability  (CJCSI 3170,  6212,  DoD  5000  &  DoD  4630.)  Each  of  these 
organizations  is  responsible  for  one  or  more  of  these  documents. 
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Revised 
Page  15 


Revised 
Page  15 


Revised 
Page  17 
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Assistant  Secretary  of  Defense  (Command, 
Control,  Communications,  and  Intelligence) 
Comments 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
6000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-6000 

August  19,  2002 

COMMAND,  CONTROL, 

COMMUNICATIONS,  AND 
INTELLIGENCE 

MEMORANDUM  FOR  THE  DOD  INSPECTOR  GENERAL 


SUBJECT:  Management  Comments  to  DOD  IG  Audit  Report  on  “Implementation  of 

Interoperability  and  Information  Assurance  Policies  for  Acquisition  of  DoD  Weapon 
Systems  (Project  No.  D2002AE-0009)” 


This  office  has  reviewed  the  subject  Audit  Report  and  concurs  overall  with  its  findings  and 
recommendations.  The  Audit  report  accurately  assesses  Department  of  Defense  progress  in 
implementation  of  interoperability  and  informadon  assurance  policies  within  the  DoD.  Provided 
below  are  our  management  comments  regarding  the  specific  findings  and  recommendations,  relevant 
to  the  Assistant  Secretary  of  Defense  (Command,  Control,  Communications),  for  this  audit: 

Recommendation  1.  This  office  concurs  with  this  recommendation  and  will  work  with  the 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition  Technology  and  Logistics  (OUSD 
(AT&L))  and  the  Joint  Staff  to  implement  a  process  for  timely  revision  and  synchronization  of  DoD 
policies  regarding  interoperability  and  information  assurance.  We  will  use  the  Interoperability 
Senior  Review  Panel  (ISRP)  to  coordinate  the  annual  review,  synchronization  and  revision,  if 
required,  for  the  following  policies  and  implementing  instructions  impacting  interoperability:  DoDD 
4630.5,  DoDI  4630.8,  CJCSI  3170.01,  and  CJCSI  6212.01.  The  ISRP  will  also  be  used  to  ensure 
consistency  of  the  above  interoperability/requirement  policies  with  the  5000  series  policy  documents 
for  acquisition;  and  DoDI  5200.40  and  CJCSI  6510.01  policies  for  information  insurance. 

Recommendation  2.  This  office  will  continue  to  work  with  the  ISRP  membership  to  refine 
the  process  for  identifying  programs  with  interoperability  deficiencies  and  their  nomination  as 
candidates  for  the  Interoperability  Watch  List.  As  the  IWL  is  a  relatively  new  oversight  tool,  we  are 
still  in  the  formative  stages  of  developing  appropriate  criteria  and  procedures  for  placing  programs 
on  the  IWL.  This  last  year,  the  Air  Force’s  Situational  Awareness  Data  Link  (SADL)  program  was 
pursued  as  initial  pilot  case  for  nomination  to  the  IWL.  The  lessons  learned  from  this  pilot  will  be 
applied  as  we  to  continue  to  explore  other  programs  for  IWL  consideration.  We  are  also  working 
with  OUSD  (AT&L),  DOT&E  and  the  Joint  Staff  to  develop  procedures  for  addressing 
interoperability  deficiencies  within  existing  processes  and  fora  prior  to  consideration  by  the  ISRP 
for  the  IWL. 

Recommendation  3.  This  office  agrees  with  the  underlying  premise  of  this  recommendation. 
However,  we  believe  it  misses  the  essential  point  that  the  Joint  Mission  Areas  and  subordinate 
supporting  mission  areas  must  first  be  defined  by  the  Joint  Staff  and  mission  area  integrated 
architectures  developed  by  DoD  Components  before  Joint  Forces  Command  can  conduct  an 
effective  comparative  analysis  operational  requirements.  Further,  DoDI  4630.8  assigns  the  Joint 
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Staff  the  responsibility  to  “5.7.4.  Ensure  mission  area  integrated  architectures,  strategies,  concepts, 
and  visions  of  the  DoD  Components  are  synchronized  to  support  IT  and  NSS  interoperability 
requirements  and  identify  opportunities  for,  and  impediments  to,  interoperability.”  Once  JMAs  and 
subordinate  mission  areas  are  defined,  the  Joint  Staff  must  coordinate  with  the  DoD  Components  to 
ensure  we  develop  a  consistent  and  integrated  set  of  mission  area  integrated  architectures  across 
DoD  Services  and  Agencies.  We  propose  that  a  specific  recommendation  be  made  in  this  audit 
report  for  the  Joint  Staff  to  further  refine  existing  JMAs  and  define  applicable  subordinate  mission 
areas  to  serve  as  the  basis  for  development  of  mission  area  integrated  architectures  by  DoD 
Components. 

Attached  are  additional  editorial  comments  to  this  audit  report  for  your  consideration. 

Should  you  have  any  questions  regarding  our  response  to  this  audit,  please  feel  free  to  contact  either 
Mr.  Jack  Zavin  at  (703)  607-0238  (jack.zavin@osd.mil)  or  Mr.  Kris  Stance  at  (703)  607-0249 
(kris.stance@osd.mil). 


Attachment: 

Additional  Comments  to  Audit  Report 
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Final  Report 
Reference 


POD  IG  Audit  Report  on  Implementation  of  Interoperability  and  Infomatioii  Assurance  Policies  for  Acquisition  of  DoD  Weapon 
Systems  (Project  D2002AE-OQ09 


# 

Page 

Paragraph 

Component 

Critical 

Substantive 

Editorial 

Comments 

1 

i 

Results 

Substantive 

Change  wording  in  the  second  sentence  from:  ..using  an  architecture  view. . .”  to:  “. .  .using  a 
mission  area  integrated  architecture. . .” 

2 

1 

■ 

ASD(C3I) 

A&I 

Directorate 

Substantive 

Change  wording  in  the  last  sentence  from:  “..  .through  interoperability  may  have 
consolidated. . .”  to:  “. ..through  interoperabihty  appears  to  have  consohdated. . 

1 

ASD(C3I) 

A&l 

Directorate 

Substantive 

Add  the  following  after  the  last  sentence:  “While  the  appropriate  tools  ate  necessary  to 
conduct  a  credible  assessment,  the  nature  of  the  JMA’s  as  defined  by  the  Joint  Staff  in  JROC 
Memo  CM-1014-00,  of  6  September  2000  are  very  high  level  and  may  contribute  to  the 
difficulties  analyzing  specific  ORD  interoperability  requirements  against  joint  mission 
architectures.  The  refinement  of  the  JMAs  and  the  definition  by  the  Joint  Staff  of  subordinate 
mission  areas  that  support  the  high  level  JMAs  is  required  to  form  a  more  complete  mission 
area  integrated  architecture  upon  which  to  base  ORD  interoperability  assessments.” 

4 

12 

2 

ASD(C3I) 

A&l 

Directorate 

Substantive 

Delete  the  second  sentence.  And  replace  with  the  following: 

“The  IWL  is  a  relatively  new  oversight  tool  and  appropriate  criteria  for  nominations  are  being 
developed.  The  Interoperability  Senior  Review  Panel  has  been  working  to  develop 
procedures  for  addressing  interoperability  deficiencies  within  existing  processes  prior  to 
consideration  for  potential  IWL  nomination. 

5 

15 

2 

Substantive 

Add  the  following  after  the  last  sentence:  “  Refining  the  JMAs,  defining  subordinate  joint 
mission  areas  and  consistent  development  and  management  of  mission  area  integrated 
architectures  by  the  DoD  Components  as  described  in  the  recently  revised  DoDD  4630.5  and 
DoDI  4630.8  represents  a  positive  step  by  the  Department  of  Defense  in  meeting  this 
challenge” 

1 


Revised 
Page  i 

Page  5 

Revised 
Page  8 

Page  12 

Page  15 
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Final  Report 
Reference 


1 

Paragraph 

Component 

Critical 

Substantive 

Editorial 

Comments 

1 

16 

3 

ASD(C31) 

A&l 

Directorate 

Substantive 

Insert  a  new  paragraph  after  paragraph  2  as  follows:  “We  recommend  that  the  Chairman  of  the 
Joint  Chiefs  of  Staff  further  refine  existing  JMAs  and  define  applicable  subordinate  mission 
areas  to  serve  as  the  basis  for  development  of  mission  area  integrated  architectures  by  DoD 
Components". 

U.S.  Joint  Forces  Command  Comments 


DEPARTMENT  OF  DEFENSE 

COMMANDER  IN  CHIEF 
U.S.  JOINT  FORCES  COMMAND 
1562  MITSCHER  AVENUE  SUITE  200 

NORFOLK.  VA  23SS1  -2488  m  iiskv  nrai  TO: 

SerJ00IG/2U9750 
19  Aug  02 


MEMORANDUM  FOR  INSPECTOR  GENERAL,  DEPARTMENT  OF  DEFENSE 

Subject:  Report  on  the  Implementation  of  Interoperability  and  Information  Assurance 
Policies  for  Acquisition  of  DoD  Weapon  System  (Project  No.  D2002AE-0009) 

1 .  The  Draft  DoD  Inspector  General  Audit  Report  on  Implementation  of  Interoperability  and 
Information  Assurance  Policies  for  Acquisition  of  DoD  Weapons  System  has  been  reviewed. 
U.S.  Joint  Forces  Command  does  not  concur  with  subject  report.  The  attachment  contains 
USJFCOM  comments  and  recommended  changes. 

2.  If  you  have  questions,  please  contact  my  Audit  Liaison,  YN2  Thaddeus  D.  Spain  at  (757) 
836-5940  or  DSN  836-5940. 


Colonel,  U.S.  Air  Force 
Inspector  General 


Attachment: 

Recommended  changes  to  Report  on  the  Implementation  of  Interoperability  and  Information 
Assurance  Policies  for  Acquisition  of  DoD  Weapon  System  (Project  No.  D2002AE-0009) 
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Final  Report 
Reference 


Revised 
Page  21 


Page  23 


Page  4 


Page  8 


Recommended  Changes  to  Report  on  the  Implementation  of  the  Interoperability  and 
Information  Assurance  Policies  of  DoD  Weapons  System  (Project  No.  D2002AE-009| 


ORG 

! 

P  Fan 

a  1 

2 

e 

! _ 

L 

i 

n 

e 

# 

Clasi 

Comments 

A/R/P 

;  JFCOM 
J86 

1 

1 

c 

E 

F 

J 

U 

Critical:  Recommendation:  As  drafted,  the  DoD  IG 
recommendation  is  not  actionable  without  Joint  Staff 
completion  of  the  Joint  Mission  Area  (JMAs)  architectures.  The 
JFCOM  recognizes  that  the  development  of  the  Joint  Mission 
Area  Architectures  is  critical  to  compare  requirements  across 
programs.  We  will  continue  to  press  this  ned  separately  and 
collectively  in  all  available  forums.  Further,  the  quantification 
analysis  and  predictive  tools  recommendation  is  also  not  fully 
actionable  at  this  time.  Although  the  software  tools  referenced 
in  the  report  are  under  JFCOM  assessment  and  show  potential, 
it  must  be  oidally  accredited  by  OSD  and  the  JS  for  the 
verification  of  joint  interoperability  requirements  in  mission 
area  architectures. 

Rationale:  Correctness. 

JFCOM 

J6 

& 

J86  : 

41  Adequacy  of  the 
Interoperability 
and  Information 
Assurance  Process 
Sub-  Paragraph  2 

U 

Critical:  Recommendation:  Change  the  paragraph  as  follows: 
“The  Commander,  USJFCOM  did  not  analyze  infonnation  assurance 

docufflentsfORDSI  does  analyze  interooerabilitv  and  information 
assurance.  However,  without  overarchine  Joint  Mission  Area 

architectures,  inteeration  reouirements  in  coordination  with 

CJCSI  3170.01  and  CJCSl  6212.01B  are  analyzed  from  known 

and  e.xistine  systems  and  as  they  intearate  with  existine  CRDs.’ 

Rationale:  The  paragraph  as  written  does  not  accurately  reflect 
the  requirements  review  process  at  JFCOM  and  the  required 
actions  as  directed  by  CJCSI  3170,01  and  CJCSI  6212.01, 

JFCOM  , 
J6  1 

8|  Interoperability 

1  Requirements 

J 

U 

Critical:  Recommendation:  Change  the  paragraph  as  follows:  ‘ 
...did  not  have  the  analytical  or  quantitative  modeling  and 

_ i 

Attachment 


74 


Final  Report 
Reference 


Recommended  Changes  to  Report  on  the  Implementation  of  the  Interoperability  and 


J86 


JFCOM 

J6 

& 

J86 


Interoperability 

Analysis 


1  simulation  tools  or  the  resources  to  conduct  that  level  of  revietv. 

!  USJFCOM  does  review  the  interonerabilitv  of  individual  system 

ORDs  within  the  framework  of  Family  of  Systems  in  existing 

'  Canstone  Reouirements  Documents  and  known  systems 

outside  of  those  ORDs  in  accordance  with  the  reouirements  of 

CJCSl  3170.01  and  CJCSl  6212.01.’ 

Rationale:  Accuracy  -  while  USFJCOM  would  like  to  be  able  to 
conduct  the  reviews  as  outlined  in  the  existing  paragraph 
(although  we  strongly  disagree  that  the  depth  of  the  01^  review 
as  outlined  in  the  report  is  currently  a  USJFCOM  mission)  to 
include  detailed  an^sis  JFCOM  does  not  have  the  resources 
(manpower  and  test  bed)  at  this  time  to  do  that. 

Critical:  Recommendation:  Delete  the  following  sentences:  ‘ ... 
contains  interoperability  KPPs  and  information  achange 
requirements.  However,  iheUSlFCOM  did  not  analyze  the  ORB 

lU^UUCinCUl«i  U^lillL;!  null  ICIillC  lU  lUiU  li^Ullliil  UUlvl  MlUnil  LAJix  | 

systenBouladelheCRDi. ...’ 

Rationale:  Accuracy.  The  above  requirement  in  the  report,  to 
review  ORDs  in  context  with  the  existing  family  of  ^sterns,  is 
not  a  JFCOM  requirement.  The  current  ORD  review  process, 
as  outlined  in  CJCSl  6212.01  and  CJCSl  3170.01,  is  to  ensure 
the  new  ORDs  interoperability  with  the  future  family  of  sj'stems 
as  outlined/described  in  the  CRDs  (the  CRD  details  the  future 
system  requirements  for  the  family  of  systems  that  include  the 
current  and  potential  innovation  resulting  form  technology  to 
be  added).  JFCOM  works  with  the  ORD  authors  who  come 
under  the  respective  JFCOM  CRDs  to  ensure  that  those  ORDs  I 
are  compliant,  as  appropriate  and  applicable,  with  the  CRDs. 
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1 

1 

1 

The  report  appears  to  not  understand  what  the  role  of  the 

CRDs  is  to  serve  as  the  road  map  to  future  interoperability  (that 
is  why  CJCSI  3170.01  and  CJCSI  6212.01  require  the 
traceability  of  ORD  requirements  to  the  CRDs  requirements! 
and  not  to  the  current/existing  systems  within  the  particular 
family  of  systems.  JFCOM  does  realize  that  additional  efforts 
are  requird  to  insure  interoperabity  and,  with  additional 
resources,  would  want  to  participate  in  those  efforts.  However, 
currently  USJFCOH  is  accomplishing  what  it  is  directed  to  do 
in  CJCSI  3170.01  and  CJCSI  6212.01.  Additionally,  the  DoD 
lacks  a  reliable  repository  of  all  existing  family  of  systems 
information. 

1  JFCOM 
:  J6 
& 

J86 

9  Interoperability 
Review  Assessment 

j 

u 

Critical:  Recommendation:  Change  the  paragraph  as  follows: 

*...  however,  the  subject  -matter  experts  do  not  compare  the 
proposed  system  ORD  against  the  ORDs  of  the  system  it  must 
be  interoperable  with,  even  if  required,  because  they  do  not  have 
the  necessary  analytical,  quantitative  modeling  and  simulation 
tools,  oersonnel.  or  the  authority  to  determine  whether  the 

ORD  is  appropriate  for  the  operational  or  systems  architecture. 
Rationale:  There  is  no  requirement  for  USJFCOM  to  do  as  the 
author  recommends  and  there  is  no  authority  for  JFCOM  to  do 
this.  The  development  of  ORDs  is  the  solely  the  role  of  the 
services  and  agencies  and  while  JFCOM  does  have  a  role  in  the 
review  process  (lAW  CJCSI  3170.01  and  CJCSI  6212.01)  the 
approv^  rests  with  the  JROC.  The  statement  is  overall  is  very 
misleading  and  does  not  in  fact  accurately  represent  this 
process.  The  author  is  again  misinterpreting  what  the  actual 
role  of  JFCOM  is  in  the  process  and  is  grossly  overstating  the 
requirements  for  JFCOM.  We  again  do  concur  that  with 
aditional  resources  there  are  many  areas  where  JFCOM  could 
improve  the  interoperability  effort,  but  these  are  areas  that  are 
currently  not  our  assigned  tasks  and  are  unfunded. 

JFCOM 

Recommendations 

U 

Critical:  Recommendation:  Add  the  new  paragraph:  ‘  Joint 

j 
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In 

ormation  Assurance  Policies  of  DoD  Weapons  System  (Project  No.  D2002AE-009I 

J6 

& 

J86 

1 

1 _ L 

6|  New  paragraph  3 

i 

i 

1 

L_ 

Forces  Command  is  currently  only  required  and  resourced  to 
review  ORDs  as  a  standalone  document  for  compliance  with  the 
CRDs  and  to  provide  a  joint  perspective.  We  recommend 
additional  resources  be  provided  to  JFCOM  to  enable  a  more  in 
depth  review  of  the  ORDs  for  their  interoperability  within  their 
respective  family  of  systems.’ 

Rationale:  JFCOM  is  currently  unfunded/resourced  to  execute 
either  recommendation  3a  or  3b. 

JFCOM 

J6 

li  Recommendations 

6  Paragraph  3a 

n 

M 

u 

Substantive:  Recommendation:  Change  the  sentence  as 
follows:  Compare  the  operational  requirements  documents  of 
proposed  DoD  systems  within  JMA  architectures,  as  thev  are 
completed,  aaainst  the  other  operational  requirement.s 
documents  in  the  related  joint  mission  area  architecture ...’ 
Rationale:  Accuracy 

JFCOM 

J86 

1  Recommendations 
6|  Paragraph 

4a 

u 

Critical:  Recommendation:  Add  a  new  paragraph  and  make  it 
paragraph  4a  as  follows:  “Provide  a  joint  perspective  as  the 
Chairman’s  advocate  for  interoperabili^  and  integration,  to 
team  with  the  Joint  Warfighting  Capability  Assessments 
(JWCAs)  in  developing  Joint  Mission  Area  Architectures.' 
Rationale:  JMA  Architectures  are  key  to  determining  the  utility 
of  systems  within  DoD.  As  the  Warfighters  advocate  for 
interoperability  and  integration,  USJFCOM  should  participate 
in  the  architecture  process  to  insure  warfighter  needs  and 
interests  are  met. 

JFCOM 

1  Recommendations 

6  Paragraph 

u 

Critical:  Recommendation:  Change  paragraph  4a  to  paragraph 
4b  and  change  the  wording  to  read,  ‘Compare  requirement 
documents  of  proposed  DoD  systems  within  JMA  Architectures, 
as  they  are  completed,  against  the  other  requirement 
documents  in  their  related  Joint  Mission  Ai^  Architecture  to 
verify  the  completeness  of  stated  interoperabity  requirements.’ 
Rationale:  JMA  Architectures  are  key  to  determining  the  utity 
of  sj'stems  within  DoD.  As  the  Warfighters  advocate  for 
interoperability'  and  integration,  USJFCOM  should  participate 
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in  the  architecture  process  to  insure  warfighter  needs  and  | 

_ j _  '  interests  are  met. _  | 

JFCOM  '  1  Recommendati^  U  Critical:  Recommendation:  Add  the  following  language  tolhe  [ 
6  Paragraph  end  of  paragraph  5, 'and  the  director,  Inte%nce,  surveiance 

and  Reconnaissance  (J2);’ 
i  i  !  Rationale:  Comoleteness 
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Director,  Operational  Test  and  Evaluation, 
Comments 


oe«ikATior«Ab  Tm 
AW  CVAA.UA'nOM 


OFFICE  OF  THE  SECRETARY  OF  DEFENSE 
1700  DEFENSE  PENTAGON 
WASHINGTON.  DC  2030M700 

AUG  2  3  2002 


MEMORANDUM  FOR  THE  DOD  INSPECTOR  GENERAL 


SUBJECT:  DoD  IG  Audit  Report  on  “Implementation  of  Interoperability  and  Information 
Assurance  Policies  for  Acquisition  of  DoD  Weapon  Systems 
(Project  No.  D2(X)2AI;-0009)“ 


I  appreciate  the  opportunity  to  review  and  comment  on  the  subject  report.  It  provides  a 
fair  representation  of  the  challenges  the  Department  of  Defense  faces  in  coordinating 
interoperability  activities  across  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  and  US 
Joint  Forces  Command.  However,  I  would  like  to  offer  the  following  comments  on  the  findings 
and  recommendations. 

Recommendation  2.  Partially  concur.  As  a  member  ofthc  Interoperability  Senior 
Review  Panel  (ISRP),  DOT&E  continues  to  work  with  the  other  members  on  refining  the 
Interoperability  Watch  List  (TWL)  processes.  The  Watch  List  is  one  of  many  tools  available 
within  the  Department  to  address  interoperability  problems.  The  IWL  policy  states  the  list  will 
be  used  for  (hose  systems  with  interopc^ility  issues  that  are  not  being  adequately  addressed  by 
other  forums.  I  believe  llie  ISRP  has  saved  a  valuable  role  over  the  last  18  months  in 
coordinating  interoperability  policies  and  actions  across  the  Department  and  working  within 
existing  forums  to  solve  issues.  For  example,  the  ISRP  worked  with  OUSD  (AT&L)  AS&C  to 
ensure  interoperability  is  stressed  within  ACTDs  to  avoid  problems  as  these  experimental  items 
transition  to  the  force.  The  interoperability  concerns  raised  by  the  ISRP  on  the  Air  Force 
Situational  Awareness  Data  Link  were  key  inputs  for  the  OASD  (C3I)  Joint  Tactical  Radio 
System  waiver  process.  The  criteria  for  IWL  candidates  suggested  by  the  report  are  certainly  a 
start,  and  DOT&E  and  the  ISRP  will  use  these  as  a  means  to  evaluate  future  candidates.  The 
report  also  cites  the  DOT&E  Annual  Report  to  Congress  as  a  source  of  systems  for  the  IWL 
Although  I  believe  these  programs  arc  already  receiving  ^equate  attention  through  T&E 
oversight  and  other  acquisition  review  processes,  if  a  program  on  T&E  Oversight  is  not 
adequately  addressing  interoperability.  I  will  bring  it  to  the  ISRP  for  consideration. 

Recommendation  S.  Partially  concur.  1  agree  that  testing  of  interoperability  and 
information  assurance  lakes  on  more  critical  status  as  we  continue  to  transform  into  a  network 
centric  force.  The  DOT&E  already  ensures  interoperability  is  tested  to  support  the  assessment  of 
Key  Performance  Parameters  provided  in  the  Operational  Requirements  Documents,  and 
published  policy  in  1999  for  testing  information  assurance.  In  2000,  DOT&E  conducted  an 
assessment  in  response  to  study  recommendations  by  the  Defense  Science  Board,  to  determine 
requirements  for  interoperability  testing  of  systems  within  the  scope  of  a  larger  networked 
system  of  systems  link^  over  the  Global  Information  Grid  (GIG).  As  a  result  of  this  effort  and 
similar  initiatives  by  OUSD  (AT&L)  and  the  Joint  Staff,  the  Department  funded  an  initial 
capability  called  the  Joint  Distributed  Engineering  Plant  (JDEP)  in  the  FY02  POM.  I  believe  the 


79 


objective  JDEP  capability  will  permit  DOTAF.  to  do  the  te^iting  called  for  in  this 
recommendation.  However,  expanded  funding  for  JDEP  is  needed  over  the  POM  before  it  will 
provide  the  full  scope  called  for  in  this  recommendation. 

Interoperability  and  information  assurance  are  key  enablers  for  the  future.  I  look  forward 
to  working  with  you  and  your  staff  on  the  follow-on  reports. 


Thomas  P.  Christie 
Director 


Defense  Information  Systems  Agency  Comments 


DEFENSE  INFORMATION  SYSTEMS  AGENCY 

701  S  COURTHOUSE  ROAD 
ARLINGTON.  VIRGINIA  22204-2199 


REFtRTO  INSPECTOR  GENERAL  (IG) 


19  August  2002 


MEMORANDUM  FOR  Department  of  Defense  Inspector  General 

SUBJECT:  Draft  Report,  Audit  of  Implementation 

Interoperability  and  Information  Assurance 
Policies  for  Acquisition  of  DOD  Weapon  Systems, 
D2002AE-0009,  FOUO 


1.  The  enclosed  document  provides  the  response  from  the  Defense 
Information  Systems  Agency  on  the  subject  Draft  report. 

2.  The  points  of  contact  for  this  action  are  Liz  Lippmann, 


Audit  Liaison,  at  703.607-6306 
at  703.607-6316. 

FOR  THE  DIRECTOR: 

Enclosure  a/s 


or  Teddie  Steiner,  Audit  Liaison, 


Quality  Information  for  a  Strong  Defense 
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JITC  COMMENTS  ON  D2002AE-0009 

Page  4  Document  page  4,  first  bullet  item: 

Background:  DoD  Interoperability  Policy  is  set  by  DoDD 

4630.5  and  DoDI  4630.8.  DoD  Regulation  5000. 2-R 
implements  the  ATSL  responsibilities  of  4630  while  CJCSI 
6212  and  3170  implement  the  Joint  Staff  responsibilities. 
CJCSI  6212.  OIB  was  in  re-write  while  4630  was  being 
revised  and  incorporated  the  intended  changes  to  4630 
prior  to  its  release  in  Jan  02. 

Draft  discussion  says  1992  version  of  ASD{C3I)  DODI 
4630.8  was  inconsistent  with  CJCSI  6212. OIB.  Appears  to 
be  attribution  of  inconsistency  to  the  wrong  source. 
ASD(C3I),  as  a  staff  element  of  OSD,  should  be  a  higher 
authority  than  the  JCS.  Hence,  criticism  could  be  that 
4630  series  were  outdated,  given  developments  in 
architecture  frameworks  and  acquisition  documents. 
Alternatively,  one  could  with  equal  validity  say  that 
6212. OIB,  in  so  far  as  it  was  in  conflict  with  4630 
series,  was  prematurely  issued,  lacking  the  authority  of 
the  4630  series  for  the  changes. 

_  ^  Document  page  5,  fourth  bullet  item: 

Page  5 

Technically,  4630.8  does  not  require  DOTSE  to  establish 
criteria.  Specifically,  it  states: 

"5.5.5.  Establish,  with  the  USD (ATSL),  the  DoD  CIO, 
the  Chairman  of  the  Joint  Chiefs  of  Staff,  and  the 
U.S.  Joint  Forces  Command,  an  interoperability  Watch 
List  (IWL)  to  provide  DoD  oversight  for  those  IT  and 
NSS  activities  for  which  interoperability  is  deemed 
critical  to  mission  effectiveness,  but 
interoperability  issues  are  not  being  adequately 
addressed.  IT  and  NSS  considered  for  the  IWL  may  be 
pre-acquisition  systems,  acquisition  programs  (any 
ACAT),  already  fielded  systems,  or  CINC-unique 
procurements . " 

The  IWL  process  is  being  addressed  by  the 
Interoperability  Senior  Review  Panel  (ISRP)  and 
there  is  a  process  identified  for  nominating  systems 
to  the  list.  POC  for  the  ISRP  would  be  Ms.  Susan 
Wright  (Susan.Wright@osd.mil) .  Systems  have  been 
nominated  for  the  list,  but  none  have  been  actually 
put  on  the  list  to  date. 
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Document  page  5,  'Interoperability  Policy': 

Recommend  re-wording  to  "The  DoD  CIO  established  policy 
guidance  concerning  interoperability  for  requirements 
generation,  acquisition  and  complete  life-cycle  of  a 
system.  USD  (ATSL)  established  the  implementation 
guidance  for  system  acquisition  and  the  Joint  Staff 
established  the  implementation  guidance  for  requirements 
certification,  supportability  certification  and  system 
interoperability  test  certification."  Rationale:  More 
complete. 

Document  page  5,  'Joint  Staff  Policy' : 

Last  sentence:  "Further,  DISA  and  the  JITC  are  to 
certify  test  results  for  all  interoperability  system 
tests."  More  correctly  should  be  "DISA/JITC."  JITC  is 
to  certify  results,  and  is,  in  fact,  designated  sole 
certifier  of  such  results  for  the  IX)D. 

Document  page  6,  'Inconsistent  Policy': 

Sentence  should  be  revised  in  line  with  the  comments  to 
page  4  and  5  above.  We  do  not  believe  that  the  DoD 
policy  is  inconsistent,  but  rather  that  the  revisions  of 
various  documents  were  not  as  coordinated  with  respect  to 
their  release  dates  as  possible.  The  various  documents 
were  coordinated  as  to  content.  They  are  now  in 
synchronization  and  are  undergoing  another  review/update 
cycle. 

Document  page  7,  'Revisions  to  Guidance' : 

Same  problem  as  identified  in  first  comment.  JCS  policy 
and  instruction  should  conform  to  OSD  guidance,  not  the 
other  way  around. 

Document  page  8,  'Interoperability  Analysis': 

Recommend  revising  since  the  ORD  review  process  does 
require  the  originator  to  identify  the  system's 
interoperability  requirements  in  the  context  of  other 
systems  and  thereby  creates  a  review  of  system  A 
requirements  vs.  system  B,  C  and  other  requirements. 

Document  page  10,  'Maintenance  of  the  Database': 


Page  5 


Revised 
Page  5 


Page  7 


Page  7 


Page  8 


Page  10 
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Page  11 


Revised 
Page  4 


Page  29 


Revised 
Page  64 


States  that  "...the  JCPAT  is  a  historical  database  that 
lists  all  DoD  systems  undergoing  interoperability 
certification  or  that  have  been  certified  for 
interoperability."  Recommend  that  this  be  revised  to 
read  "...the  JCPAT  is  a  historical  database  that  lists 
all  DoD  systems  undergoing  interoperability  requirements 
certification  or  that  have  had  their  interoperability 
requirements  documents  certified."  Rationale:  More 
accurate  as  the  JCPAT  is  a  repository  of  interoperability 
requirements  documents. 

Document  page  10,  'Access  to  Database' : 

Third  sentence  says  756  users,  15  contractor  support 
personnel.  Fourth  sentence  says  DISA  could  identify 
total  number  of  users,  but  could  not  distinguish  DOD  from 
contractor  personnel.  Both  statements  cannot  be  correct 
as  they  stand. 

Document  page  11,  first  bullet  item: 

Phrase  'interoperability  certification'  is  ambiguous. 

The  JS  issues  interoperability  certifications  of 
requirements  and  supportability  documents  (two  separate 
types  of  certifications),  and  issues  system  validation 
memoranda.  JITC  issues  system  interoperability  test 
certifications  based  on  interoperability  testing. 

Document  page  18,  last  sentence: 

Paragraph  deals  entirely  with  information  assurance 
issues.  End  of  last  sentence  reads:  "...  to  verify  that 
the  systems  had  an  information  assurance  strategy  and 
were  certified."  Correct  term  to  reflect  approval  is 
"...to  operate  in  a  particular  security  mode  using  a 
prescribed  set  of  safeguards  at  an  acceptable  level  of 
risic"  with  respect  to  DITSCAP  procedures  is  'accredited' 
vice  'certified.' 

Document  page  48: 

Distribution:  Should  read  Commander,  Joint 
Interoperability  Test  Command 


84 


Joint  Staff  Comments 


THE  JOINT  STAFF 
WASHINGTON,  DC 


Reply  ZIP  Code:  DJSM-806-02 

20318-0300  03  September  2002 


MEMORANDUM  FOR  THE  INSPECTOR  GENERAL,  DEPARTMENT  OF 
DEFENSE 

Subject:  Report  on  the  Implementation  of  Interoperability  and  Information 

Assurance  Policies  for  Acquisition  of  DOD  Weapon  Systems  (Project 
No.  D2002AE-0009) 

1.  Thank  you  for  the  opportunity  to  review  and  comment'  on  the  subject  draft 
report.  1  concur  subject  to  incorporation  of  the  enclosed  comments. 

2.  The  Joint  Staff  point  of  contact  is  Lieutenant  Colonel  Scott  Hoffman,  614- 
7004. 

JOHN  P.  ABIZAID 
Lieutenant  General,  USA 
Director,  Joint  Staff 

Enclosure 

Reference: 

1  DOD  IG  memorandum,  19  July  2002,  "Report  on  the  Implementation  of 
Interoperability  and  Information  Assurance  Policies  for  Acquisition  of 
DOD  Weapon  Systems  (Project  No.  D2002AE-0009)’' 
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Page  4 


Page  4 


Page  8 


ENCLOSURE 

JOINT  STAFF  COMMENTS  ON  REPORT  ON  THE  IMPLEMENTATION  OF 
INTEROPERABILITY  AND  INFORMATION  ASSURANCE  POLICIES  FOR 
ACQUISITION  OF  DOD  WEAPON  SYSTEMS 
(PROJECT  NO.  D2002AE-0009) 

1.  Page  4.  2d  bullett.  Change  as  follows:  “The  Commander,  USJFCOM 
did  not  onalyge  interoperability  and  information  assurance  requirements 
in  the  operational  and  system  architectures  for  DoD  eyotemo  as  port  of 
the  interoperability  certification  process^  for  operational  requirements 
documents  (ORDS)^  does  analyze  interoperability  and  information 
assurance.  However,  without  overarching  Joint  Mission  Area 
architectures,  integration  requirements  in  coordination  with  CJCSl 
3170.01  and  CJCSI  6212. OIB  are  analyzed  from  known  and  existing 
systems  as  they  integrate  with  existing  CRDs.” 

REASON:  The  paragraph  does  not  accurately  reflect  the  requirements 
reyiew  process  at  USJFCOM  and  the  required  actions  as  directed  by 
CJCSI  3170.01  and  CJCSI  6212.01. 

2.  Page  4.  3d  bullet.  Change  as  follows:  “The  Director,  Command, 
Control,  Communications,  and  Computer  Systems  Directorate  (J-6)  (the 
Joint  Staff  J-6)  had  not  updated  the  Joint  Command,  Control, 
Communications,  Computer,  and  Intelligence  Program  Assessment  Tool 
(JCPAT)  database  to  include  all  interoperability  certifioatione’  Test  and 
Evaluation  Mooter  Plana  (TEMP6);-and  -GemmQnd,  Contrel; 
Communioationo,  and  Intelligence  (C3I)  support  piano  for  DoD  sytems 
with  documents  that  ore-date  JCPAT.  Because  these  documents  are  in 
hard  copy  form,  they  do  not  facilitate  interoperability  certification.” 

REASON:  In  response  to  the  statement  that  JCPAT  did  not  include  all 
interoperability  certifications,  it  should  be  noted  that  there  are  certified 
documents  and  programs  that  pre-date  JCPAT.  The  Joint  Staff  J-6 
maintains  a  hard  copy  of  programs  and  documents  that  pre-date  JCPAT. 

3.  Page  8,  Interoperability  Requirements,  1st  paragraph,  second 
sentence.  Change  as  follows:  “...and  making  recommendations  to  the 
Chairman  of  the  Joint  Chiefs  of  Staff  on  mieoion  need  statements  and 
operational  requirements  documents  requirement  documents.” 

REASON:  These  recommendations  apply  to  all  requirement 
documents.  All  references  to  operational  requirements  documents 
(ORDs)  in  this  report  should  be  changed  to  read  requirement  documents. 
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4.  Page  8.  Interoperability  Reouirenients.  1st  paragraph,  last  sentence. 
Change  as  follows:  “...did  not  have  the  analytical  or  quantitative 
modeling  and  simulation  tools  or  the  resources  to  conduct  that  level  of 
review.  USJFCOM  does  review  the  interoperability  of  individual  system 
ORDs  within  the  framework  of  Family  of  Systems  in  existing  Capstone 
Requirements  Documents  and  known  systems  outside  of  those  ORDs  in 
accordance  with  the  requirements  of  CJCSl  3170.01  and  CJCSl 
6212.01.'’ 

REASON:  Accuracy.  While  USFJCOM  would  like  to  conduct  the 
reviews  as  outlined  in  the  existing  paragraph  (USJFCOM  disagrees  that 
the  depth  of  the  ORD  review  as  outlined  in  the  report  is  currently  a 
USJFCOM  mission),  to  include  detailed  analysis,  USJFCOM  does  not 
have  the  required  resources  (manpower  and  test  bed). 

5.  Page  8.  2d  paragraph.  Change  as  follows:  “...contains 
interoperability  KPPs  and  information  exchange  requirements.  However; 
tbe-  USJFCOM  did  not  onalyge  the  ORD  requirements  of  a  proposed  BoP 
system  within  the  context  of  ORD  requirements  of  other  oyoteme  with 
which  a  proposed  BoD  system  must  operate  and  exchange  informatioBr 
Instead  of  analyaing  the  ORDs  that  the  system  will  interoperate  with,  the 
USJFCOM  stated  that  it  reviews  ORDs  within  the  context  of  CRDs  and 
measuree  -intefoperability  and  integration  requirements  against  ORDs 
that  relate4e  CRDs  and  against  other  known  BoD  systems  outside  the 
CRDc  ” 

REASON:  Accuracy.  The  above  requirement  in  the  report,  to  review 
ORDs  in  context  with  the  existing  family  of  systems,  is  not  a  USJFCOM 
requirement.  The  current  ORD  review  process,  as  outlined  in  CJCSl 
6212.01  and  CJCSl  3170.01,  is  to  ensure  the  new  ORDs  interoperate 
with  the  future  family  of  systems  as  outline  and  described  in  the  CRDs 
(the  CRD  details  the  future  system  requirements  for  the  family  of 
systems  that  include  the  current  and  potential  innovations  resulting 
form  technology  to  be  added).  USJFCOM  works  with  the  ORD  authors  to 
ensure  that  those  ORDs  are  compliant,  as  appropriate  and  applicable, 
with  the  Capstone  requirements  documents  (CRDs).  The  role  of  the 
CRDs  is  to  serve  as  a  road  map  to  future  interoperability  (that  is  why 
CJCSl  3170.01  and  CJCSl  6212.01  require  the  traceability  of  ORD 
requirements  to  CRDs  requirements)  and  not  to  current  and  existing 
systems  within  the  particular  family  of  systems.  USJFCOM  does  realize 
that  additional  efforts  are  required  to  insure  interoperability  and,  with 
additional  resources,  would  participate  in  those  efforts.  However,  it  is 
accomplishing  what  was  directed  under  CJCSl  3170.01  and  CJCSl 
6212.01. 
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Page  9 


Page  10 


Page  17 


6.  Page  9.  Interoperability  Review  Assessment.  2d  sentence.  Change  as 
follows:  “However,  the  subject  matter  experts  do  not  compare  the 
proposed  system  ORD  against  the  ORDs  of  the  system  it  must  be 
interoperable  with,  even  if  required,  because  they  do  not  have  the 
necessary  analytical,  quantitative  modeling  and  simulation  tools  have 
not  been  approved  or  provided.  Although  USJFCOM  has  the 
responsibility  to  recommend  ORD  changes  that  enhance  interoperability, 
USJFCOM  has  neither  the  personnel  or  the  authority  to  determine 
whether  the  ORD  is  appropriate  for  the  operational  or  systems 
architecture.". 

REASON:  There  is  no  requirement  or  authority  for  USJFCOM  to  do  as 
the  author  recommends.  The  development  of  ORDs  is  solely  the  role  of 
the  Services  and  agencies.  While  USJFCOM  does  have  a  role  in  the 
review  process  (CJCSl  3170.01  and  CJCSl  6212.01)  the  approval  rests 
with  the  JROC.  The  statement  as  drafted  is  misleading  and  does  not 
accurately  represent  the  Joint  Staff-supported  process.  The  actual  role 
of  USJFCOM  is  only  within  the  process. 

7.  Page  9.  Tools  paragraph.  Comment:  The  recommended  tool  is  under 
development.  It  is  not  DOD  accepted,  approved  or  certified.  USJFCOM 
could  evaluate  the  utility  of  this  or  any  other  tool  with  respect  to  the  role 
that  USJFCOM  performs  in  the  requirements  process  prior  to  approving 
any  language  this  specific. 

8.  Page  10,  Maintenance  of  the  Database.  2d  paragraph.  1st  sentence. 
Change  as  follows:  “When  searching  for  interoperability  certification 
documents  in  the  JCPAT,  ACAT  11  or  below  documents,  a  user  could  not 
be  assured  that  the  document  located  must  be  aware  of  the  correct 
milestone  for  the  program  to  determine  the  most  recently  recent  certified 
document.” 

REASON:  Requirements  documents  are  submitted  at  each  milestone. 
Depending  upon  the  milestone,  there  may  be  more  than  one  document 
on  a  particular  program.  A  reviewer  would  have  to  be  aware  of  the 
milestone  and  the  date  processed  to  determine  the  current  document. 

We  agree  that  the  user  cannot  be  assured  that  the  document  is  the  most 
recently  approved  or  validated  document.  However,  this  is  only  true  for 
ACAT  II  and  below  documents.  Since  the  Joint  Requirements  Oversight 
Council  is  a  validating  authority,  the  J-8  post-validated  ACAT  1  or  JSI 
programs  to  the  J-8  tool.  Since  both  tools  use  the  same  repository,  the 
validated  documents  are  available  in  both  the  J-6  and  J-8  tool.  Recent 
changes  to  J6  JCPAT  should  alleviate  confusion. 

9.  Page  16.  Recommendations,  end  of  paragraph  1 .  Add:  “We  also 
recommend  the  implementation  of  a  process  which  addresses,  funds. 
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and  implements  aspects  of  joint  battle  management  command  and 
control  interoperability  and  connectivity,  validation  and/or  allocation  of 
resources  to  acquire  Joint  systems  and  create  offices  supporting 
integration  of  materiel  and  non-materiel  solutions  for  broad  mission 
capability  areas." 

REASON:  The  IG  report  does  not  address  the  fundamental  issue  that 
the  Department  of  Defense  is  not  effectively  structured  to  effect  the 
“organizing,  training,  and  equipping”  of  joint  capabilities.  There  is  no 
joint  process  responsible  and  accountable  for  developing  and  acquiring 
joint  command  and  control  systems  and  integrating  capabilities. 

10.  Page  16.  Recommendations,  paragraph  2.  Add  the  following 
subparagraphs: 

“a.  Develop  Joint  Mission  Area  Architectures  based  on  currently 
available  POD  assets  and  based  on  those  assets  needed  to  provide  the 
capabilities  for  the  future. 

b.  Provide  an  analysis  branch  within  each  JMA  to  assess  Requirement 
Document  relevance  against  known  architecture  needs. 

c.  Develop  compliant,  certified,  and  standard  analysis  and  predictive 
tools,  such  as  models  or  simulations,  to  assist  in  verifying  the 
completeness  of  stated  interoperability  requirements  in  mission  area 
architectures.” 

REASON:  Completeness. 

11.  Page  16.  Recommendations,  paragraph  3.  Comment:  USJFCOM  is 
currently  required  to  only  review  ORDs  as  stand-alone  documents  for 
compliance  with  CRDs.  The  command  does  not  possess  the  authority  or 
resources  to  provide  a  more  in-depth  review  of  ORDs  for  interoperability 
with  the  respective  family  of  systems. 

12.  Page  16.  Recommendations,  subparagraph  3a.  Change  as  follows: 
“Compare  all  relevant  the  operational  requirements  documents  of 
proposed  DoD  IT  and  NSS  fto  include  SAPl  systems  within  JMA 
architectures,  as  they  are  completed,  against  all  other  operational 
requirements  documents  in  the  related  joint  mission  area  architectures 
to  verify  the  completeness  of  stated  interoperability  requirements.” 

REASON:  Accuracy. 

13.  Page  16.  Recommendations,  paragraph  4.  Change  as  follows:  “We 
recommend  that  OASD.  PISA,  the  Director.  Command,  Control, 
Communications,  and  Computer  Systems  Directorate  (J-6),  Office  of  the 
Chairman  of  the  Joint  Chiefs  of  Staff,  and  USJFCOM:". 
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Revised 
Page  25 


Revised 
Page  50 


REASON:  JCPAT  access  and  archiving  are  functional  responsibilities 
of  DISA.  JCPAT  is  also  used  by  OSD.  The  recommendation  should  be 
addressed  to  all  users,  to  include  USJFCOM.  JMA  Architectures  are  the 
key  to  determining  the  utility  of  systems  within  the  Department  of 
Defense.  As  the  warfighters  advocate  for  interoperability  and  integration, 
USJFCOM  should  participate  in  the  architecture  process  to  insure 
warfighter  needs  and  interests  are  met. 

14.  Page  16.  Recommendations,  subparagraph  4b.  Change  as  follows: 
“In  conjunction  with  other  coordination  between  OASD.  DISA  and  Joint 
Staff  directorates,  limit  Joint  Command,  Control,  Communications, 
Computers,  andjntelligence  Program  Assessment  Tool  access  to  users 
who  have  a  need  to.know.” 

REASON:  JCPAT  is  a  collaborative  database  used  by  at  least  four 
different  executive  directorates  tmd  agencies.  (J-6/J-8/OASD/DISA). 
Since  the  J-6  tool  is  only  one  element  of  JCPAT,  this  statement  should  be 
directed  to  include  all  users. 

15.  Page  42.  Appendix  G.  Change  diagram  as  follows:  Draw  a  line 
directly  from  JCPAT  (a)  to  J-6  (b)  and  add  a  line  between  J-8  (a)  and 
USJFCOM  (d). 

REASON:  The  diagram  does  not  distinguish  between  ACAT  I,  JSI  and 
ACAT  II  and  below  systems.  The  diagram  also  indicates  that  J-6  receives 
documents  from  J-8.  This  not  true  for  ACAT  II  and  below.  ACAT  II  and 
below  programs  are  staffed  directly  from  JCPAT.  Additionally, 
documents  are  forwarded  to  USJFCOM  from  both  J-8  tmd  J-6. 
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Interoperability  Senior  Review  Panel  Comments 


OFFICE  OF  THE  SECRETARY  OF  DEFENSE 

I  CX»  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-1000 

2  7  AUG 

MEMORANDUM  FOR  THE  DOD  INSPECTOR  GENERAL 

SUBJECT;  DoD  IG  Audit  Report  on  “Implementation  of  Interoperability  and  Information 
Assurance  Policies  for  Acquisition  of  DoD  Weapon  Systems 
(Project  No.  D2002AE-0009r 


The  Interoperability  Senior  Review  Panel  GSRP)  has  reviewed  the  subject  report.  The 
ISRP  consists  of  senior  leaders  from  the  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition  Technology  and  Logistics  (OUSD  (AT&L)),  the  Office  of  the  Assistant  Secretary  of 
Defense  (Command,  Control.  Communications  and  Intelligence  (OASD  (C3I)),  the  Joint  Staff. 
Joint  Forces  Command,  Office  of  the  Director  for  Programs,  Analysis  and  Evaluation  (PA&E) 
and  Director,  Operational  Test  and  Evaluation  (DOT&E).  Recognizing  that  the  primary 
objective  of  the  report  was  an  evaluation  of  Defense  and  the  Defense  Agencies  implementation 
of  DoD  mteroperabiUty  and  information  assurance  policies,  the  IG  report  does  not  address  the 
fun^ent^  issue  that  DoD  is  not  currently  structured  to  effect  “organizing,  training,  and 
equipping  required  joint  capabilities.  The  Services  determine  requirements  and  resource  those 
requirements  based  on  individual  Service  vice  Joint  priorities.  Further,  IT  and  NSS 
interoperability  and  information  assurance  should  be  viewed  in  a  wider  context  of  Doctrine 
Organization,  Training,  Materiel,  Leadership  Development,  Persormel,  and  Facilities 
(DOTMLPF)  instead  of  only  system  of  system  and/or  family  of  systems.  Viewed  in  this  context 
joint  interoperability  requires  a  synchronized  effort  and  resources  across  the  entire  DOTMLPF 
spectrum.  The  ISRP  offers  the  following  specific  comments  on  the  recommendations: 

Becommendation  1 .  The  ISRP  concurs  with  this  recommendation  and  will  work  with  the 

Staff  to  implement  a  process  for  timely  revision  and  synchronization  of  DoD  policies  regarding 
interoperability  and  information  assurance.  The  ISRP  membership  will  be  used  to  coordinate  the 
annual  review,  synchronization  and  revision,  if  required,  for  the  following  policies  and 
implementing  instructions  impacting  interoperability:  DoDD  4630.5,  DoDI  4630.8,  CJCSI 
3170.01,  and  CJCSI  6212.01.  The  ISRP  will  also  coordinate  with  OUSD  (AT&L),  0/VSD  (C31) 
Md  the  Joint  Staff  to  ensure  consistency  of  the  above  interoperability/requirements  policies  with 
the  5000  series  policy  documents  for  acquisition,  and  DoDI  5200.40  and  CJCSI  65 1 0.01  policies 
for  infonnation  insurance.  ^ 

Recommendation  2.  The  ISRP  will  continue  to  refine  the  process  and  procedures 
regarding  implementation  of  the  Interoperability  Watch  List.  Our  pilot  case  this  past  year  was 
the  Air  Force  Situational  Awareness  Data  Link  (SADL).  SADL  was  based  on  a  non-Joint 
Tactical  Radio  System  compliant  radio,  using  a  non-standard  air-to-air  and  air-to-ground  tactical 
data  link.  The  ISRP  determined  the  interoperability  issues  were  significant.  The  visibility  the 
ISRP  brought  to  the  SADL  allowed  the  existing  OASD  (C3I)  JTRS  Waiver  process  to  proceed 
with  the  decision  to  disapprove  the  SADL  request  and  have  the  Air  Force  design  an  interoperable 
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solution  for  its  close  air  support  aircraft.  The  ISRP  continues  to  investigate  other  programs  for 
the  Interoperability  Watch  List  and  will  work  to  find  solutions  within  established  processes  of 
the  Joint  Staff,  OUSD  (AT&L),  OASD  (C3I),  and  DOT&E  before  placing  a  system  on  the  List. 

Recommendation  3.  The  ISRP  concurs  with  the  concept  of  the  basic  recommendation 
and  recognizes  the  development  of  the  Joint  Mission  Area  Architectures  is  critical  to  compare 
requirements  across  programs.  We  will  continue  to  press  this  need  separately  and  collectively  in 
all  available  forums.  However,  as  drafted,  the  recommendation  is  not  actionable  by  JFCOM 
without  Joint  Staff  completion  of  the  Joint  Mission  Area  (JMAs)  architectures.  Further,  the 
quantification  analysis  and  predictive  tools  recommendation  is  also  not  fully  actionable  at  this 
time.  Although  the  software  tools  referenced  in  the  report  are  under  JFCOM  assessment  and 
show  potential,  it  must  be  officially  accredited  by  OSD  and  the  JS  for  JFCOM  to  provide 
verification  of  joint  interoperability  requirements  in  mission  area  architectures. 

Recommendation  4.  The  ISRP  supports  the  position  of  the  Joint  Staff  regarding  this 
recommendation.  The  responsibility  for  JCPAT  goes  beyond  the  Joint  Staff,  so  that  the 
recommendation  should  be  addressed  to  all  departments,  agencies,  and  directorates  that  either 
use  or  conhol  access  to  JCPAT.  Since  many  members  of  the  ISRP  benefit  from  JCPAT,  the 
ISRP  will  look  into  the  policy  and  guidance  associated  with  JCPAT  and  make  recommendations 
to  appropriate  organizations. 

Recommendation  5.  The  ISRP  agrees  that  testing  of  interoperability  and  information 
assurance  takes  on  more  critical  status  as  we  continue  to  develop  and  transform  into  a  network 
centric  force.  The  DOT&E  currently  tests  interoperability  to  ensure  the  Key  Performance 
Parameters  within  the  Operational  Requirements  Documents  are  met,  and  has  a  policy  for  testing 
information  assurance.  The  ISRP  supports  the  development  of  necessary  test  capabilities  to 
represent  relevant  aspects  of  the  Global  Information  Gnd  that  will  allow  programs  to  test  in  a 
system  of  systems  environment.  OUSD  (AT&L),  OASD  (C3I),  and  DOT&E  support  expanded 
funding  in  the  upcoming  POM  for  the  Joint  Distributed  Engineering  Plant  which  will  allow  for  a 
cost  effective  means  to  do  this  type  of  testing. 

The  ISRP  point  of  contact  is  Ms  Susan  Wright,  Executive  Secretary,  703-681-1440 
extension  115. 


George  G.  Wauer 
Co-Chair 
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